Unverified50% confidenceBenchmarkExact time
The best-performing frontier flagship model achieved a chance-corrected Cohen's κ of about 0.29 against expert labels, far below the human ceiling of about 0.53.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
11/3/2026
First Seen
Valid until: 11/17/2026
Sources
Related Entities
Related Claims
UnverifiedIn the setting closest to reality (where unstated constraints must be actively discovered), the best-performing model configuration achieves an average FDE-Gym score of only 0.56.74% similarUnverifiedIn the example, for the same set of human labels (18 pass, 2 fail), a lazy judge that passes everything gets an accuracy of 0.90 but a kappa of 0.00; a better judge gets an accuracy of 0.95 but a kappa of only 0.77, still not reaching the 0.80 threshold.70% similarUnverifiedOn four Theory-of-Mind benchmarks, the weak model's average score rose from 0.49 to 0.91.70% similarUnverifiedMid-tier models typically achieve more than 80% of flagship model scores on benchmarks such as MMLU and HumanEval, while requiring less than 30% of the inference cost.70% similarUnverifiedThe Spearman correlation coefficient between tool description scores and correct model selection was only 0.04, with almost no statistical association.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1129267API
curl https://kongchang.com/api/v1/knowledge/claims/1129267MCP
get_claim(id=1129267)