Unverified50% confidenceCautionExact time
LMArena's blind-test voting mechanism has been pointed out by researchers to have a prompt engineering bias, whereby models that are good at generating fluent, flattering responses score higher rather than those with genuinely stronger capabilities.
1
Sources
50%
Confidence
Long-term
Relevance
9/25/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe overconfidence of a model answering directly while skipping retrieval is technically referred to as miscalibration, meaning there is a systematic deviation between confidence and actual accuracy.74% similarUnverifiedWhen multiple LLMs share similar pre-training data or alignment biases, majority voting may fail to correct shared biases, which is why the study additionally introduced independent human blind-test validation.70% similarUnverifiedResearch reveals that LLMs do not reliably weight and update based on evidence reliability like an ideal Bayesian agent, but instead exhibit behavioral patterns close to distributional skew, with their influence being less correlated with the true reliability of evidence than the Bayesian framework would predict.69% similarUnverifiedDenser relevance judgments reduce the probability that correct results retrieved by a model are mistakenly judged as irrelevant, thereby improving evaluation reliability.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/949535API
curl https://kongchang.com/api/v1/knowledge/claims/949535MCP
get_claim(id=949535)