Unverified50% confidenceOpinionExact time
The team realized that their scoring-criteria calibration was in fact training people to align with existing judgments and hiding where the criteria were ambiguous.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedArena mechanisms based on human blind evaluation and real-task benchmarks can largely circumvent the overfitting (teaching to the test) problem of static benchmarks.69% similarUnverifiedUsing a public evaluation framework means that in theory anyone can reproduce the results following the same process, thereby enhancing the credibility and transparency of the scores.69% similarUnverifiedSensitivity stemming from the evaluation method may arise from the scoring rubric favoring certain phrasings, a single reference answer format, or an automatic evaluator that parses output format too rigidly.68% similarUnverified人类评估者往往对流畅、自信、符合直觉的回答给予更高评分，这是奉承偏差（Sycophancy）的形成根源68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1064062API
curl https://kongchang.com/api/v1/knowledge/claims/1064062MCP
get_claim(id=1064062)