Unverified50% confidenceFactExact time
Evaluation runs are performed in memory; when a use case declares no expectation the scorer reports "not measured" rather than awarding full marks, and the six synthetic use cases cannot establish accuracy.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/11/2026
First Seen
Valid until: 1/9/2027
Sources
Related Entities
Related Claims
Verified若用含测试集的全量数据计算均值和标准差，会导致预处理泄漏，使评估指标虚高，无法真实反映模型泛化能力73% similarUnverifiedStandardized benchmarks have the limitation of out-of-distribution generalization, where agents may score by memorizing training set patterns rather than truly possessing reasoning ability.71% similarUnverified如果参数量充足的模型连几十个样本都无法记住，几乎可以断定存在硬性错误，如标签错位、损失函数计算错误、梯度未正确回传等71% similarUnverifiedIn class-imbalanced scenarios, accuracy alone can be misleading and needs to be combined with a confusion matrix for more fine-grained evaluation.70% similarUnverifiedVague task descriptions, lack of clear acceptance criteria, undefined scope of modifications, and failure to decompose sub-steps can cause agents to fall into repeated exploratory loops.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/998474API
curl https://kongchang.com/api/v1/knowledge/claims/998474MCP
get_claim(id=998474)