Unverified50% confidenceOpinionExact time
When benchmark scoring metrics become optimization targets, a Goodhart's Law effect arises: models learn to score high on specific formats without necessarily truly improving their capabilities, and some models with HumanEval scores exceeding 95% vary dramatically in actual code generation ability.
1
Sources
50%
Confidence
Long-term
Relevance
7/23/2026
First Seen
Sources
Qwen3 235B A22B实测：2.4万亿参数开源模型超越Claude Opus
Related Claims
UnverifiedBenchmark overfitting is a core problem in current large model evaluation; some models achieve pass rates above 90% on HumanEval but perform far below expectations in real codebases.71% similarUnverifiedEven the most advanced code generation models achieve a functional correctness rate of only 60%–80% for generated code, while the models' confidence in the correctness of their own output is often as high as 90% or more.69% similarUnverified模型在标准化评测基准（如HumanEval、MMLU）上的高分表现与真实工作场景中的可靠性之间存在显著差距68% similarUnverified模型的基准分数更高并不等于在所有场景下都更好用67% similarUnverifiedResearch indicates that high-quality prompts can improve model output accuracy by 30%–50%.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/600091API
curl https://kongchang.com/api/v1/knowledge/claims/600091MCP
get_claim(id=600091)