Unverified50% confidenceFactExact time
SPLIT's formula explains a counterintuitive phenomenon: under certain conditions, adding more LLM judges actually reduces accuracy, because verifier errors are correlated, and calling the same or similar verifiers multiple times cannot independently eliminate errors and may instead amplify systematic bias.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
Verified若用含测试集的全量数据计算均值和标准差，会导致预处理泄漏，使评估指标虚高，无法真实反映模型泛化能力78% similarUnverifiedarXiv paper 2609.22512 points out that treating the consensus of multiple LLM reviewers as strong evidence presupposes that the errors of each reviewer are mutually independent, whereas in reality LLM reviewers make the same errors due to similar training data and evaluation approaches, causing consensus to exaggerate evidential strength.77% similarUnverifiedThe breaking points of automated evaluation include: LLM evaluators are lenient, judge partially correct answers poorly, are sensitive to prompt wording, miss small errors, and their agreement rates can mask low recall.76% similarVerifiedLLMs may make incorrect judgments due to hallucinations, prompt injection, or contextual misunderstanding, and without independent verification mechanisms, faulty reasoning can directly translate into erroneous production operations76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1065992API
curl https://kongchang.com/api/v1/knowledge/claims/1065992MCP
get_claim(id=1065992)