Unverified50% confidenceFactExact time
In 2023, the authors of MT-Bench proposed having the evaluator answer the question itself first, then using its own answer as the reference answer, which reduced the failure rate on math problems from 70% to 15%.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAmong the five evaluators the authors tested, if the answer key had minor errors, the score was only 0.21 — lower than giving no key at all (0.46) and lower than giving a completely irrelevant key (0.50); while a human-verified model-generated answer scored 0.61, close to the 0.69 of a human-written one.68% similarUnverifiedAmong 19 studies that reported which direction the evaluator's errors lean, 12 lean toward passing work that should have failed, 3 lean in the opposite direction, and 4 are mixed.67% similarUnverifiedPlacing verbose padding on the correct answer causes the judge's score to plummet—Grok non-reasoning hard-question scores dropped from 0.875 to 0.19—indicating that verbosity bias exists only in reverse.67% similarUnverifiedIn university-level proof grading, even with the experts' grading rubric in hand, 5 of 7 evaluators still passed 38.0% to 45.7% of proofs that experts judged as failures, while falsely failing only 5.0% to 12.3% of correct proofs.66% similarUnverified评分器要求每个金额与标准答案的误差在£0.005以内，前提标志正确，且桥接表在适用处必须对账成功，哪怕错1便士也判定为失败66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1026026API
curl https://kongchang.com/api/v1/knowledge/claims/1026026MCP
get_claim(id=1026026)