Unverified50% confidenceOpinionExact time
When evaluation focuses only on model benchmark scores, system-level reliability differences are masked.
1
Sources
50%
Confidence
Long-term
Relevance
10/27/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedA single unsafe execution rate metric masks capability differences among models, so evidence acquisition behavior and post-recovery decision quality need to be reported as well.79% similarVerifiedIn benchmark testing, execution failures and model decisions should be counted separately to avoid misinterpreting a single quota overrun as a deficiency in model capability.78% similarVerified若用含测试集的全量数据计算均值和标准差，会导致预处理泄漏，使评估指标虚高，无法真实反映模型泛化能力77% similarUnverifiedDenser relevance judgments reduce the probability that correct results retrieved by a model are mistakenly judged as irrelevant, thereby improving evaluation reliability.76% similarUnverifiedOptimization fidelity analysis can only verify predefined transformation patterns and cannot discover optimizations invented by the model that are not listed in the benchmark; some semantically equivalent transformations may appear different on the AST, leading to missed detection.76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1102673API
curl https://kongchang.com/api/v1/knowledge/claims/1102673MCP
get_claim(id=1102673)