Unverified50% confidenceTradeoffExact time
The approach of building gold labels based on consensus among multiple judge models carries the risk of systematic bias, as judge models sharing similar training data or value alignment may reach consensus on the same class of errors in a way that is difficult to detect through voting mechanisms.
1
Sources
50%
Confidence
Long-term
Relevance
10/5/2026
First Seen
Sources
Related Entities
Related Claims
Unverified多评判者投票系统在多个存在不同偏差的评判者独立评分时偏差方向相互抵消，聚合结论可靠性高于单一评判者71% similarUnverifiedThere is a risk of verification credibility issues in multi-agent systems: agents may falsely report test passes due to misunderstanding, hallucination, or execution failure.68% similarUnverifiedA common limitation shared by mainstream bias benchmarks is that they measure a model's explicit choices in specific controlled contexts, rather than implicit biases that emerge in open-ended generation, long-text reasoning, or complex role-playing scenarios.68% similarUnverifiedWhen evaluating fact-checking systems, evidence quality and answer correctness should be accounted for separately, as a single joint score can easily obscure the true capability boundaries.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/975453API
curl https://kongchang.com/api/v1/knowledge/claims/975453MCP
get_claim(id=975453)