Unverified50% confidenceFactExact time
Existing evaluation benchmarks tend to fix scenarios and phrasing, allowing models to pass tests by memorizing standard answers while failing in real interactions where phrasing varies—a phenomenon known as benchmark overfitting.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedWhen automated scores fluctuate greatly but manual review finds the quality of multiple variant answers to be comparable, the problem lies in the evaluation stage rather than the model.79% similarUnverified同样的模型只改几句提示词，回答效果可能天壤之别，提示词设计本质上是持续测试、不断调整、反复优化的循环79% similarUnverified上下文回贯缺失指工具执行后的观测结果（observation）没有回写到对话上下文，导致模型每轮接收相同上下文而重复相同决策77% similarUnverifiedOverfitting refers to a model being repeatedly tuned on a validation or test set until it memorizes the characteristics of that data rather than genuinely learning generalizable capabilities76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1000972API
curl https://kongchang.com/api/v1/knowledge/claims/1000972MCP
get_claim(id=1000972)