Unverified50% confidenceFactExact time
After a retry succeeds, the real observation of the failure (e.g., a 503) is discarded and only the final clean receipt is retained, meaning you paid twice but kept only the worse fact.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedFailed retries across multiple models may accumulate actual costs, and if only the cost of the successful attempt is recorded, billing attribution becomes distorted.70% similarUnverifiedDue to sampling variance, each test message should be run twice; a rule that holds with only 50% probability is effectively no rule at all.69% similarUnverifiedTreating every failure as an equally weighted deduction is a serious evaluation pitfall, as an average score of 95% can mask the most costly and irreversible failures within that 5%.68% similarUnverifiedScoring with completion-style proxy metrics will classify failures that terminate cleanly but produce incorrect results as successes, which is a flaw in many current agent evaluations.67% similarUnverifiedReplay-based validation can detect 76.2% of unsuccessful improvements, whereas build-check-only approaches fail to detect any of these behavioral failures.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1003285API
curl https://kongchang.com/api/v1/knowledge/claims/1003285MCP
get_claim(id=1003285)