Unverified70% confidenceOpinionExact time
Mainstream benchmark problems have been widely discussed on the internet, and models' training data may contain memorization of these problems, causing benchmark scores to overestimate the models' capabilities on truly unseen tasks
2
Sources
70%
Confidence
Long-term
Relevance
7/6/2026
First Seen
Sources
GPT-5.5 Codex Reasoning-Token Clustering Explained: Causes of Performance Regression and How to Respond
hackernewshackernews7/4/2026
Related Claims
UnverifiedThe self-built benchmark contains a large number of proprietary questions not included in mainstream large model training data, enabling it to reflect models' generalization ability rather than memorization ability.76% similarUnverifiedModels trained at large scale may acquire meta-cognitive information from training data about 'being an AI' and 'potentially being under evaluation'.75% similarUnverifiedOpenAI在InstructGPT论文中证明，少量高质量人类偏好数据对模型对齐的贡献远超大规模无监督训练75% similarVerifiedBenchmarks face contamination risks; if training data contains test questions or approximate variants, a model's high scores may partly stem from memorization rather than genuine capability.75% similarUnverifiedThe accuracy of data format conversion directly affects the subsequent model training effectiveness for object detection.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/123359API
curl https://kongchang.com/api/v1/knowledge/claims/123359MCP
get_claim(id=123359)