Unverified50% confidenceFactExact time
In the adjudication queue, there were cases showing one reviewer marking partial compliance as a pass while another marked it as a fail.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe probe includes a pair of fake judges as a self-check: one is set to give different scores to the harm version (should be flagged), and the other gives the same score to both versions (should not be flagged), otherwise the run report is invalid.78% similarUnverifiedMandatory checks should be separated from scoring; no matter how high the score on optional check items, it cannot offset a failed mandatory check.75% similarUnverifiedMost benchmarks use accuracy as the core metric, counting both wrong answers and refusals as incorrect and not distinguishing the difference in harm.68% similarUnverified评估集应来自真实疑难案例（缺失发票、已付款发票、不匹配账户、政策例外等），而非十个简单的理想路径案例67% similarUnverifiedWhen using LLM-as-a-judge for safety evaluation, one should explicitly check the judge's refusal rate rather than only looking at the final score.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1064063API
curl https://kongchang.com/api/v1/knowledge/claims/1064063MCP
get_claim(id=1064063)