Unverified60% confidenceTradeoffExact time
There is a disconnect between high benchmark scores and actual user experience—a model that scores highly on all tests may come across as rigid and inflexible in everyday conversations.
2
Sources
60%
Confidence
Long-term
Relevance
9/9/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedFactors such as context accumulation and excessively long sessions can affect model performance, making it difficult to distinguish subjective experience from actual performance regression.76% similarUnverified人类标注者在评估答案时存在信息越丰富越好的认知偏差，导致奖励模型将篇幅与质量隐性挂钩，是模型冗余的成因之一71% similarUnverifiedSoft capabilities such as conversational naturalness, knowledge accuracy, and genuine helpfulness are difficult to reflect in evaluation leaderboards but are most strongly felt in everyday use.71% similarVerifiedThere is a gap between benchmark scores and real user experience, and some vendors engage in targeted optimization for specific benchmarks (score gaming) as well as training data contamination that inflates scores.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/881629API
curl https://kongchang.com/api/v1/knowledge/claims/881629MCP
get_claim(id=881629)