Unverified50% confidenceFactExact time
The questions and answers of public benchmarks (such as MMLU and HumanEval) are usually publicly available, so model teams can specifically cover similar questions in their training data to obtain inflated scores on the leaderboards, that is, "benchmark gaming."
1
Sources
50%
Confidence
Long-term
Relevance
10/28/2026
First Seen
Sources
DeepSeek V5 Code Leak? It's Actually a Compatibility Test
bilibili阿梨Aria早鸟报10/28/2026
Related Entities
Related Claims
UnverifiedMainstream evaluation sets such as MMLU, HumanEval, and GSM8K have long been publicly available on the internet, making it highly likely that models have been exposed to these questions during pretraining or fine-tuning.79% similarUnverifiedMainstream benchmark problems have been widely discussed on the internet, and models' training data may contain memorization of these problems, causing benchmark scores to overestimate the models' capabilities on truly unseen tasks76% similarUnverifiedEnsuring that training data contains a sufficient proportion of genuine human-original content has become a key issue in large model development, and companies such as OpenAI and Anthropic invest substantial resources in data filtering and quality control for this purpose.73% similarUnverifiedSpecialized models (classifiers, rerankers, entity models) are usually easier to calibrate than an LLM's self-reported confidence, because their training objective directly optimizes the correspondence between scores and correctness.73% similarUnverifiedOpenAI在InstructGPT论文中证明，少量高质量人类偏好数据对模型对齐的贡献远超大规模无监督训练72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1109726API
curl https://kongchang.com/api/v1/knowledge/claims/1109726MCP
get_claim(id=1109726)