Unverified50% confidenceBenchmarkExact time
On science questions answered based on the literature, the best evaluator, o3, agreed with the human vote 65.1% of the time, while random guessing is 50%.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/14/2026
First Seen
Valid until: 1/12/2027
Sources
Related Entities
Related Claims
Unverified研究表明人类评估者在判断LLM输出的事实准确性时，正确率往往只有50%-70%73% similarUnverifiedSelf-Consistency通过多次采样推理路径后投票取多数答案，可进一步提升准确率约5-10个百分点68% similarUnverifiedIn time-series direction forecasting, the correct baseline should be the majority class proportion rather than 50%, because if 51% of movements are upward, 'always guess up' achieves 51% accuracy.67% similarUnverifiedUnder the condition of n=100 with an even split of positive and negative samples, the 95% confidence interval for 55% accuracy is roughly 45%–65%, which covers the 50% baseline, so the 5 percentage point gap may be random fluctuation.67% similarUnverifiedPapers that were randomly boosted with initial selections saw their probability of subsequent selection increase by 45.55 percentage points (95% confidence interval: 41.20 to 49.90).66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1026033API
curl https://kongchang.com/api/v1/knowledge/claims/1026033MCP
get_claim(id=1026033)