Unverified50% confidenceBenchmarkExact time
Multiple studies from 2024 indicate that the agreement rate between LLM judges and human experts is typically between 70% and 85%.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
8/7/2026
First Seen
Valid until: 11/5/2026
Sources
从生产事故到评估用例：LLM回归测试的实战方法论
Related Claims
UnverifiedResearch shows that the agreement rate between LLM judges and human scores is approximately 80%, approaching but not yet surpassing the level of human inter-annotator agreement (approximately 81%).78% similarUnverified2023年学术界研究发现，顶级大模型作为评判者与人类专家的判断一致性可达80%以上75% similarUnverified研究表明LLM裁判与人类专家的一致性在许多任务上可达80%以上75% similarUnverified斯坦福HAI研究所2023年报告显示，最新LLM在HumanEval等代码基准测试中正确率已超过85%，远超2021年的不足50%73% similarVerifiedThe LLM-as-Judge method can achieve over 80% agreement with human expert scoring.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/705571API
curl https://kongchang.com/api/v1/knowledge/claims/705571MCP
get_claim(id=705571)