Unverified50% confidenceFactExact time
In inter-annotator agreement assessment, only a Cohen's kappa or Krippendorff's alpha above 0.6 counts as substantial agreement, and only above 0.8 counts as near-perfect agreement.
1
Sources
50%
Confidence
Long-term
Relevance
11/8/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedCohen's κ > 0.6 is generally considered substantial agreement, and 0.4–0.6 moderate agreement.75% similarUnverifiedFor annotation consistency, Cohen's kappa was 0.781 (range 0.753-0.807) and Fleiss' kappa was 0.77.73% similarUnverifiedInter-annotator agreement (IAA) is commonly calculated using Cohen's Kappa coefficient (for two annotators) and Fleiss' Kappa coefficient (for multiple annotators), where a Kappa value of 1 indicates perfect agreement and 0 indicates agreement no better than chance.69% similarUnverified事实性标注通常要求Kappa大于0.8,而主观性标注可能只要求Kappa大于0.668% similarUnverifiedOn the disagreement-enriched dataset, the Cohen's kappa agreement coefficient between the gpt-4o-mini judge and the gold standard jointly confirmed by two authors was only 0.0468% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1147637API
curl https://kongchang.com/api/v1/knowledge/claims/1147637MCP
get_claim(id=1147637)