Unverified50% confidenceFactExact time
Humans have a self-consistency rate of about 66% on binary preference tasks, human annotation costs about $300 per 1,000 items, and LLM annotation is about 50 times cheaper and has a higher agreement rate with the human majority opinion.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/17/2026
First Seen
Valid until: 1/15/2027
Sources
Related Entities
Related Claims
Unverified研究表明人类评估者在判断LLM输出的事实准确性时，正确率往往只有50%-70%67% similarUnverifiedThe author recommends designing the Agent's confidence threshold as follows: execute autonomously above 90%, present an intermediate state for user confirmation between 60% and 90%, and below 60% forcibly acknowledge uncertainty without guessing.67% similarUnverified在歧义点的四类行为中，'先搜再问'成功率最高，平均通过率达93.4%，远高于'直接猜'的56.5%，'搜多次再猜'为51.5%66% similarUnverifiedThe benchmark produces two numbers for each model that are never merged: a task score and a false-confidence rate (the proportion of cases that should have been escalated but were answered anyway).65% similarUnverifiedOn science questions answered based on the literature, the best evaluator, o3, agreed with the human vote 65.1% of the time, while random guessing is 50%.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1051007API
curl https://kongchang.com/api/v1/knowledge/claims/1051007MCP
get_claim(id=1051007)