Unverified50% confidenceFactExact time
In classification tasks with imbalanced classes, accuracy can be misleading; for example, if only 20 out of 1000 tickets are truly urgent, a model that predicts 'not urgent' for all of them can still achieve 98% accuracy.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedWhen samples are severely imbalanced, Accuracy becomes misleading—a model that predicts everyone as healthy can still achieve an accuracy of over 99%.77% similarUnverifiedRanked by question difficulty, persistence rate was 91.4% on the 60 questions that all models answered correctly, but only 73.5% on the 52 questions that at least one model got wrong, indicating that sycophancy tracks uncertainty.74% similarUnverifiedThe benchmark produces two numbers for each model that are never merged: a task score and a false-confidence rate (the proportion of cases that should have been escalated but were answered anyway).74% similarUnverifiedWhen the sample size is small (e.g., only 5–10 successful requests), quantile estimation errors can be large; numpy.percentile or a larger sample size can be used to improve reliability.74% similarUnverifiedIn class-imbalance scenarios, a model can achieve seemingly impressive accuracy by always predicting the majority class, a phenomenon known as the Accuracy Paradox.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1011110API
curl https://kongchang.com/api/v1/knowledge/claims/1011110MCP
get_claim(id=1011110)