Unverified50% confidenceDecision RuleExact time
The practical effect threshold (Minimum Detectable Effect, MDE) requires engineers to define in advance how large a regression constitutes a genuine problem — for example, triggering a block only when the success rate drops by more than 5 percentage points.
1
Sources
50%
Confidence
Long-term
Relevance
9/27/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedSimply setting a fixed threshold (e.g., blocking whenever the success rate drops by more than X%) will continuously trigger false alarms due to the normal fluctuation of agents, making the CI process unreliable.69% similarVerifiedDrift detection involves a trade-off: setting the threshold too sensitively leads to frequent false alarms and unnecessary retraining, while setting it too loosely may allow model performance to continuously degrade without being detected.69% similarUnverifiedAggregate metrics (such as multi-task average scores) have an inherent ability to dilute failures; when simpler tasks constitute the majority, the average score can mask significant degradation on a small number of high-difficulty tasks.68% similarUnverifiedAI Agent多步推理中,每一步5%~10%的性能抖动会逐级放大,导致整体成功率断崖式下滑,P95延迟可翻5到10倍68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/953023API
curl https://kongchang.com/api/v1/knowledge/claims/953023MCP
get_claim(id=953023)