Expired50% confidenceFactTime unknown
AI benchmark suites like MMLU, HumanEval, and MATH may produce inflated scores due to benchmark contamination, where models have indirectly encountered test data during training.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
Valid until: 9/30/2026(expired)
Sources
Gemini 3.1 Pro vs Claude Opus 4.6: Five Real-World Tests to Determine the Winner
bilibiliAI大龙虾
Related Claims
UnverifiedScale AI在2024年的分析显示，部分模型在MMLU上的分数膨胀可能高达5-10个百分点，因测试集已泄露到训练数据中72% similarUnverified模型在标准化评测基准（如HumanEval、MMLU）上的高分表现与真实工作场景中的可靠性之间存在显著差距66% similarPartially VerifiedBenchmark contamination refers to a model having already seen test questions during training, resulting in inflated scores that fail to reflect true generalization ability.66% similarUnverifiedCurrent top AI models still score far below human expert levels on CTF benchmarks, especially in challenge categories involving dynamic debugging and binary analysis.66% similarUnverifiedA 2024 study indicates that even when AI-generated content accounts for only 10–20% of training data, measurable quality degradation can occur after multiple generations of training.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/55594API
curl https://kongchang.com/api/v1/knowledge/claims/55594MCP
get_claim(id=55594)