Unverified50% confidenceSolutionExact time
The control condition is used to test whether judges can distinguish differences in Agent behavior: in the two versions the Agent behavior differs but the outcome is the same, and a correct judge should give different scores.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe core differentiating design of this tool is scoring the judge itself, comparing the judge's scores against real-world signals (user complaints, actual reported results) to determine whether the problem lies with the Agent or the judge.72% similarUnverifiedWhen evaluating Agents, it is necessary to distinguish between correctness and consistency; measuring with external correctness labels is more meaningful than simply counting terminal-state consistency.71% similarUnverifiedIf the judge only reads the agent's self-report rather than the actual artifacts, a run that did everything wrong can still pass, so the judge must reason from independently verified facts and have veto power.71% similarUnverifiedHaving multiple Agents offer opinions from different perspectives and cross-review each other (consultation) can reduce the risk of single-point misjudgment71% similarUnverifiedPaired human evaluation is a preference evaluation method: human judges are shown the outputs of two systems for the same task simultaneously and choose the better one, and the win rate is then computed; it measures relative preference rather than absolute correctness.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1043799API
curl https://kongchang.com/api/v1/knowledge/claims/1043799MCP
get_claim(id=1043799)