Unverified50% confidenceCautionExact time
Evaluation sets suffer from representational bias, as they can only cover situations the builders anticipated, meaning a model that performs perfectly on the eval set may still fail on real traffic after launch.
1
Sources
50%
Confidence
Long-term
Relevance
10/3/2026
First Seen
Sources
Related Entities
Related Claims
Verified若用含测试集的全量数据计算均值和标准差，会导致预处理泄漏，使评估指标虚高，无法真实反映模型泛化能力76% similarUnverified在功能尚未完全成熟的阶段引入外部测评可能会放大潜在问题的曝光度76% similarUnverified为几个百分点的榜单差距进行定向优化是本末倒置，评估模型时不应盲目迷信榜单排名75% similarVerifiedSome models carry the risk of over-optimization toward test sets, causing leaderboard scores to deviate from real-world performance (benchmark saturation).74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/971066API
curl https://kongchang.com/api/v1/knowledge/claims/971066MCP
get_claim(id=971066)