Judge Model
In AI agent systems, a small model used to independently judge whether an agent's behavior has fallen into a loop or anomaly; it is typically lighter and cheaper than the main agent model.
Timeline (last 90 days)
Failure-loop intervention uses a separate "judge model" to determine whether the agent is stuck in a repetitive loop, and the judge model only gives hints rather than forcibly terminating the task.
Position bias means the judge model's judgment of information in the context is affected by where that information appears: content near the end is often given higher weight, while key warnings that appear early are diluted in long conversations.
A controlled "source migration" experiment showed that judges tend to accept erroneous values visible in the input while rejecting correct values whose source is hidden, i.e., judges are "following the reference" rather than independently verifying the correctness of values.
The researchers proposed the DeRef (Derived Reference) scheme: an independent model reads the full conversation history (but does not look at the candidate value) to generate reference text for the judge, and the judge then makes its judgment based on the reference text.
Do not let the judge simultaneously take on the two responsibilities of "derivation" and "verification"; assigning the derivation step to a dedicated model and letting the judge focus on comparison can significantly improve the reliability of the entire evaluation pipeline.
If the judge systematically favors values that are "documented" and rejects correct values that "require reasoning," then correct parameters that require complex reasoning to derive may be incorrectly marked as failures in evaluation.
Tool-calling judge models have a systematic flaw: when no source for a value can be found in the input, the judge is more inclined to reject it even if the value is completely correct, a phenomenon named the "derivation gap".
Cloud services can invoke independent judge models, retrieval-augmented systems, or multi-model voting to cross-validate outputs.
All Facts (8)
Failure-loop intervention uses a separate "judge model" to determine whether the agent is stuck in a repetitive loop, and the judge model only gives hints rather than forcibly terminating the task.
50%UnverifiedPosition bias means the judge model's judgment of information in the context is affected by where that information appears: content near the end is often given higher weight, while key warnings that appear early are diluted in long conversations.
50%UnverifiedA controlled "source migration" experiment showed that judges tend to accept erroneous values visible in the input while rejecting correct values whose source is hidden, i.e., judges are "following the reference" rather than independently verifying the correctness of values.
50%UnverifiedThe researchers proposed the DeRef (Derived Reference) scheme: an independent model reads the full conversation history (but does not look at the candidate value) to generate reference text for the judge, and the judge then makes its judgment based on the reference text.
50%UnverifiedDo not let the judge simultaneously take on the two responsibilities of "derivation" and "verification"; assigning the derivation step to a dedicated model and letting the judge focus on comparison can significantly improve the reliability of the entire evaluation pipeline.
50%UnverifiedIf the judge systematically favors values that are "documented" and rejects correct values that "require reasoning," then correct parameters that require complex reasoning to derive may be incorrectly marked as failures in evaluation.
50%UnverifiedTool-calling judge models have a systematic flaw: when no source for a value can be found in the input, the judge is more inclined to reject it even if the value is completely correct, a phenomenon named the "derivation gap".
50%UnverifiedCloud services can invoke independent judge models, retrieval-augmented systems, or multi-model voting to cross-validate outputs.
50%