Unverified50% confidenceFactExact time
Aggregate metrics (such as average scores) mask failures in long-tail scenarios; when 95% of routine cases perform excellently, the collapse of the remaining 5% of edge cases leaves no obvious mark on the overall score.
1
Sources
50%
Confidence
Long-term
Relevance
8/7/2026
First Seen
Sources
从生产事故到评估用例：LLM回归测试的实战方法论
Related Claims
VerifiedReAct's errors amplify in a chain: if the accuracy per step is p, the overall success rate after n steps is approximately p to the power of n; with a per-step accuracy of 95%, the overall success rate drops to approximately 59.9% after 10 steps and only 35.8% after 20 steps.68% similarVerifiedIn long-chain tasks, if each step has an error rate of approximately 5%, the overall success rate may drop to below 36% after 20 steps.67% similarVerifiedIn scenarios with extreme class imbalance, even an accuracy as high as 95% may indicate that the model is completely unable to identify the minority class.67% similarUnverifiedTreating every failure as an equally weighted deduction is a serious evaluation pitfall, as an average score of 95% can mask the most costly and irreversible failures within that 5%.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/705566API
curl https://kongchang.com/api/v1/knowledge/claims/705566MCP
get_claim(id=705566)