Unverified50% confidenceFactExact time
Multiple academic studies have shown that models scoring highest on mainstream benchmarks such as MMLU and HumanEval often perform significantly worse on novel questions of equivalent difficulty.
1
Sources
50%
Confidence
Long-term
Relevance
9/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedSince the second half of 2024, the score improvements of mainstream large models on standard benchmarks such as MMLU and HumanEval have narrowed noticeably.80% similarUnverified模型在标准化评测基准（如HumanEval、MMLU）上的高分表现与真实工作场景中的可靠性之间存在显著差距78% similarVerifiedOn mainstream evaluations such as MMLU, HumanEval, and GSM8K, the score gaps among leading large models are often only a few percentage points.77% similarUnverified基准污染（Benchmark Contamination）导致HumanEval、MMLU等主流基准区分度下降，部分模型HumanEval得分超过95%、MMLU超过90%但实际能力平庸76% similarUnverifiedThe score gap among multiple frontier models on MMLU has narrowed to the order of one percentage point, driving the emergence of more challenging benchmarks such as MMLU-Pro and ARC-AGI.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/910051API
curl https://kongchang.com/api/v1/knowledge/claims/910051MCP
get_claim(id=910051)