Unverified50% confidenceFactExact time
LLM-as-a-Judge has three known biases: a tendency to prefer longer answers, greater tolerance for the outputs of its own model series, and sensitivity to position (options appearing earlier are more likely to be selected).
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedLLM-as-a-judge模式即用一个能力更强的模型来评判另一个模型的输出质量，降低评测成本但引入评判模型自身的偏见问题78% similarUnverifiedWhen using LLM-as-Judge, it is best for the judge and the generating Agent to use different models or different system prompts, in order to avoid shared biases causing them to endorse each other.77% similarUnverifiedThe phenomenon where classification models are sensitive to the order of candidate answers is called position bias, where candidates appearing near the end of a sequence sometimes receive higher weights in unidirectional autoregressive models.75% similarUnverifiedScoring functions often require careful design and may even rely on another LLM for scoring, known as the LLM-as-judge pattern.75% similarUnverifiedLLM裁判仅作辅助，不可单独作为最终评判依据，缓解策略包括多模型交叉打分、结构化评分Rubric、定期与人工标注对齐72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1036734API
curl https://kongchang.com/api/v1/knowledge/claims/1036734MCP
get_claim(id=1036734)