Unverified50% confidenceFactExact time
After MMLU's discriminative power declined, the community shifted toward more difficult benchmarks such as GPQA (a graduate-level professional question set), SWE-bench, MATH-500, and LiveBench.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
8/4/2026
First Seen
Valid until: 11/2/2026
Sources
Related Claims
UnverifiedSince the second half of 2024, the score improvements of mainstream large models on standard benchmarks such as MMLU and HumanEval have narrowed noticeably.68% similarUnverifiedTraditional benchmarks such as MMLU and HumanEval have experienced benchmark saturation due to training data contamination; when GPT-4 was released, its MMLU score was approximately 86%, whereas multiple open-source models have now surpassed 90%68% similarUnverified基准污染（Benchmark Contamination）导致HumanEval、MMLU等主流基准区分度下降，部分模型HumanEval得分超过95%、MMLU超过90%但实际能力平庸68% similarUnverifiedOn mainstream evaluations such as MMLU, HumanEval, and GSM8K, the score gaps among leading large models are often only a few percentage points.68% similarUnverifiedBy mid-2024, top models had generally exceeded scores of 85–90% on MMLU, causing the benchmark's discriminative power to drop sharply.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/684193API
curl https://kongchang.com/api/v1/knowledge/claims/684193MCP
get_claim(id=684193)