Unverified50% confidenceSolutionExact time
LLM-as-a-judge gatekeeping is robust at ranking agents with large capability gaps, but is least reliable when screening top candidates of similar ability.
1
Sources
50%
Confidence
Long-term
Relevance
9/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedResearch shows that both classical automated metrics and reference-free LLM-as-a-Judge methods have proven unreliable under large-scale expert judgment.71% similarUnverifiedLLMs are far less effective than humans as judges and validators, which is currently their greatest weakness.69% similarUnverifiedAlthough reference-free LLM scoring is convenient, it may systematically deviate from expert judgment.67% similarUnverifiedTreating LLMs as genuine intelligence leads to overestimating their reliability, mistakenly trusting their reasoning capabilities, and lacking vigilance regarding the hallucination problem.66% similarUnverified单一LLM智能体承担过多角色时，会因目标冲突产生决策偏差，表现为自我确认偏误66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/919806API
curl https://kongchang.com/api/v1/knowledge/claims/919806MCP
get_claim(id=919806)