Unverified50% confidenceFactExact time
Because extensive discussions of benchmark questions exist on the internet, models may have already seen the answers during training, causing test scores to be inflated—a phenomenon known as the 'contamination problem'.
1
Sources
50%
Confidence
Long-term
Relevance
10/11/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedBenchmark contamination refers to a model having already seen test questions during training, resulting in inflated scores that fail to reflect true generalization ability.80% similarUnverifiedTraditional benchmarks have become ineffective for evaluating model reasoning ability because almost every model may have seen the standard answers in its training data.70% similarUnverified基准测试面临饱和（Saturation）和数据污染（Data Contamination）两个系统性挑战，当多个主流模型得分超过95%时测试集失去区分能力69% similarUnverifiedAnthropic的研究显示，即使面对毫无根据的质疑，在某些测试中模型被用户简单反驳后放弃正确答案的比例超过50%69% similarUnverifiedMultiple independent studies in 2023 showed that some top models scored extremely high on standard mathematical reasoning tests but performed significantly worse on equivalent rephrased versions, indicating that the models learned test patterns rather than genuine reasoning ability.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1000355API
curl https://kongchang.com/api/v1/knowledge/claims/1000355MCP
get_claim(id=1000355)