LLM-as-a-Judge

LLM-as-a-judge模式即用一个能力更强的模型来评判另一个模型的输出质量，降低评测成本但引入评判模型自身的偏见问题

Means to mitigate the judge's own unreliability include multi-judge cross-checking, blinding of situation labels, pre-validation of situation definitions, and sandbox isolation; production deployment also requires judge confidence scores, manual review of low-confidence cases, and A/B testing of judge versions.

The LLM judge itself may make mistakes, including misjudging correct behavior as a violation, missing real violations, and giving inconsistent judgments for similar situations; the paper validates the judge using manually verified synthetic golden conversations, but this is costly and hard to scale.

Requiring the LLM judge to reason first and then score can significantly improve accuracy.

In LLM judge scenarios, kappa can reveal bias that accuracy masks—for example, when the judge passes everything, accuracy may be high, but kappa will be close to 0, indicating that the judge has no discriminative ability.

LLM judges should not be the only evaluation method, because they are costly, have randomness and may be fooled by the Agent's output style.

LLM judges need clear scoring dimensions (such as factual accuracy, completeness and safety), and should provide a few labeled examples and structured output (score and rationale) for human auditing.

Using a model as a judge to read each claim and citation one by one and determine whether it supports it, and having a second agent rebut the findings, resulted in 45% of the findings being overturned.

The LLM judge model inherits the biases in its own training data and may systematically prefer a certain tone, length, or style of expression, so the audit framework records the judge model name as a separate field to achieve transparency.

When an LLM scores as a judge, even with temperature set to 0, the same conversation trajectory and judge model run five times still produces a different verdict once, and this non-determinism is unacceptable for a test suite that wants to be integrated into CI.

The evaluation tool uses a hybrid approach: deterministic keyword/structural matching as the primary signal, with the LLM as judge only as a secondary layer used to evaluate the quality of free-text reasoning; the judge must cite the exact trajectory fragment on which its verdict is based, otherwise the verdict is automatically discarded.

Hallucination guardrails can be automated through an LLM judge: an auxiliary small LLM acts as a judge in the pipeline, comparing the final answer against the retrieved text before it reaches the user and deciding whether to let it through or block it; when the check fails, it can choose to retry retrieval or directly return 'I don't know'.

The judge LLM itself can also make mistakes, so its decisions need to be sampled and reviewed, and the judge model likewise needs to be evaluated and calibrated; it cannot be assumed to always be correct.

Probabilistic judgments (such as whether an answer is harmful, or whether it cites a document that does not exist) have no single correct answer and must rely on model semantic understanding, with cost and latency several orders of magnitude higher.

The official documentation gives a rough figure: with 100,000 requests per month and 10% sampling, a mini-level judge costs roughly 2 to 10 USD per month of the organization's own provider spending.

Current mainstream Agent evaluation approaches fall into three categories: task completion rate based on deterministic assertions, LLM-as-Judge, and end-to-end benchmark test sets (such as SWE-Bench and WebArena).