Unverified50% confidenceOpinionExact time
Dan Klein argues that layering a second model on top of a system to check the first is a weak fix, because both can err and often fail on the same hard cases; he estimates that 80% checking 80% ends up at about 82%.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedDan Klein argues that a system with about 80% or 90% accuracy is still unreleasable if one in ten customers is told a flight is booked when it is not; a system that is correct every time in 70% of scenarios is more valuable.65% similarUnverifiedReplay-based validation can detect 76.2% of unsuccessful improvements, whereas build-check-only approaches fail to detect any of these behavioral failures.64% similarUnverifiedScores should be weighted by failure severity rather than simply tracking pass rates; a system with a 92% pass rate where 8% are catastrophic failures is worse than a system with a 70% pass rate that degrades gracefully.63% similarUnverifiedIn the TRACES test, every model had a failure rate above 71% on the 'agentic' probes; 22 of the 30 models had failure rates above 90%.62% similarUnverified一个 90% 成功、10% 出错的工具调用系统几乎无法投入实际使用，因为单次错误会污染后续上下文造成级联失败62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1062301API
curl https://kongchang.com/api/v1/knowledge/claims/1062301MCP
get_claim(id=1062301)