Unverified50% confidenceFactExact time
Models such as GPT-4, Claude, and Gemini successively broke the 90% accuracy threshold on MMLU, causing the benchmark to lose its discriminative power.
1
Sources
50%
Confidence
Long-term
Relevance
9/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedMMLU scores jumped from approximately 70% for GPT-3.5 to 86% for GPT-4, and then to above 90% for subsequent models.77% similarUnverifiedTop autoregressive models such as GPT-4o and Claude 3.5 have already exceeded 95% accuracy on GSM8K, leading to benchmark saturation.76% similarUnverifiedResearchers including Tal Ullman at New York University found that GPT-4's performance dropped sharply when classic theory-of-mind tests were modified slightly, suggesting the model may be performing surface-level pattern matching.71% similarUnverifiedOn mainstream benchmarks such as MMLU, HumanEval, and GSM8K, open-source models have reached 85–95% of the level of early versions of GPT-4.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/910486API
curl https://kongchang.com/api/v1/knowledge/claims/910486MCP
get_claim(id=910486)