Unverified50% confidenceBenchmarkExact time
In the evaluation, gpt-4o judging had the highest agreement with the author's own 115 blind judgments (κ 0.86), and CSR judgments matched human judgment on 25 to 27 of 30 blind cases.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/17/2026
First Seen
Valid until: 1/15/2027
Sources
Related Entities
Related Claims
UnverifiedManual review of 177 verifier judgments shows an accuracy of 97.7%.71% similarUnverifiedIn the example, for the same set of human labels (18 pass, 2 fail), a lazy judge that passes everything gets an accuracy of 0.90 but a kappa of 0.00; a better judge gets an accuracy of 0.95 but a kappa of only 0.77, still not reaching the 0.80 threshold.71% similarUnverifiedThe gpt-4o-mini judge mislabeled as high as 77.1% of human cases on the enriched set that should have been judged as faithful67% similarUnverifiedIn the Phase 1 post-hoc comparison, SAGE's evaluation results were compared against 105 human reviews from the original competition, yielding an ordinal consistency of kappa = 0.29, which falls in the "fair" range.67% similarUnverifiedA document evidence model agreed with the reference labels on 36 out of 40 observations, for an accuracy of 90%.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1051800API
curl https://kongchang.com/api/v1/knowledge/claims/1051800MCP
get_claim(id=1051800)