Unverified50% confidenceDecision RuleExact time
On data with extremely imbalanced classes, accuracy rewards a judge for agreeing on the easy majority class, whereas Cohen's kappa corrects for the possibility of chance agreement, so kappa should be used rather than accuracy.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAverage accuracy masks distributional skew; a judge that gets all easy cases right and is overconfident on hard cases can post a high average score while failing at critical points.78% similarUnverifiedThe core property of a strictly proper scoring rule is that the expected score is maximized if and only if the forecaster reports the probability they truly believe, with common examples including log-likelihood loss and the Brier score.74% similarUnverifiedIn class-imbalance scenarios, a model can achieve seemingly impressive accuracy by always predicting the majority class, a phenomenon known as the Accuracy Paradox.72% similarUnverifiedPaired comparison is more reliable than independent evaluation because it eliminates randomness introduced by sample selection.71% similarUnverifiedIn the example, for the same set of human labels (18 pass, 2 fail), a lazy judge that passes everything gets an accuracy of 0.90 but a kappa of 0.00; a better judge gets an accuracy of 0.95 but a kappa of only 0.77, still not reaching the 0.80 threshold.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1035671API
curl https://kongchang.com/api/v1/knowledge/claims/1035671MCP
get_claim(id=1035671)