Unverified50% confidenceTradeoffExact time
Scores should be weighted by failure severity rather than simply tracking pass rates; a system with a 92% pass rate where 8% are catastrophic failures is worse than a system with a 70% pass rate that degrades gracefully.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedTreating every failure as an equally weighted deduction is a serious evaluation pitfall, as an average score of 95% can mask the most costly and irreversible failures within that 5%.79% similarUnverifiedSetting an absolute failure count ceiling for small-sample slices is more robust than setting a percentage threshold and is insensitive to sample size76% similarUnverified一个 90% 成功、10% 出错的工具调用系统几乎无法投入实际使用，因为单次错误会污染后续上下文造成级联失败75% similarUnverified传统的成功率指标难以全面反映策略的鲁棒性，一个策略可能在90%的常规任务中表现出色却在剩余10%的关键场景中出现灾难性失败74% similarUnverifiedScoring with completion-style proxy metrics will classify failures that terminate cleanly but produce incorrect results as successes, which is a flaw in many current agent evaluations.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1000837API
curl https://kongchang.com/api/v1/knowledge/claims/1000837MCP
get_claim(id=1000837)