Unverified50% confidenceOpinionExact time
There exists an evaluation-application gap between benchmark performance and actual production environment performance, and some vendors tend to over-optimize for evaluation sets (benchmark gaming / benchmark contamination).
1
Sources
50%
Confidence
Long-term
Relevance
8/31/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedThere is a gap between benchmark scores and real user experience, and some vendors engage in targeted optimization for specific benchmarks (score gaming) as well as training data contamination that inflates scores.76% similarUnverified基准的元评估指标包括区分度、构建效度和生态效度，缺乏生态效度的基准可能产生与真实业务表现脱节的评估结论70% similarUnverified真正被业界认可的模型排名必须经得起他人用相同方法复现，厂商内部评测因缺乏透明度易出现测试集污染或挑题展示问题69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/829046API
curl https://kongchang.com/api/v1/knowledge/claims/829046MCP
get_claim(id=829046)