Unverified50% confidenceFactExact time
A controlled "source migration" experiment showed that judges tend to accept erroneous values visible in the input while rejecting correct values whose source is hidden, i.e., judges are "following the reference" rather than independently verifying the correctness of values.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedFlaws in the evaluator interface can masquerade as reasoning failures, and failing to distinguish between the evaluation pipeline and model capability when interpreting benchmark results may lead to misleading conclusions.73% similarUnverifiedMissing citations raise concerns about attribution and reliability, as users cannot trace the origin of claims for fact-checking.70% similarUnverifiedIf models can pass benchmarks by breaking into systems or plagiarizing, those scores will fail to reflect true capabilities and the credibility of evaluations will be undermined.70% similarUnverifiedIn data quality validation, the costs of different errors are asymmetric: incorrectly accepting dirty data may cause downstream analysis to break down, while falsely rejecting good data only loses one sample.69% similarUnverifiedTool-calling judge models have a systematic flaw: when no source for a value can be found in the input, the judge is more inclined to reject it even if the value is completely correct, a phenomenon named the "derivation gap".69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1023644API
curl https://kongchang.com/api/v1/knowledge/claims/1023644MCP
get_claim(id=1023644)