Unverified50% confidenceFactExact time
AI judges show outcome bias when scoring: when only the run outcome is changed (while the Agent's decision process is the same), the AI judge gives a lower PROCESS score in the version where the outcome is wrong.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAs a probabilistic model, the AI reviewer Agent's pass judgment on the same output is random, and multiple reviews may yield different conclusions.73% similarUnverifiedA 35% merge rate means that AI-submitted changes passed review under the same standards applied to human code, rather than being accepted by lowering the bar.72% similarUnverifiedThe 80% rule means that when an AI's confidence in a judgment reaches or exceeds 80%, the system directly executes the result (writing it back to the data); when it falls below this threshold, the case is handed over for human review.70% similarUnverifiedThe scoring method was a weighted detection score: clearly describing a bug earned full marks, merely mentioning it earned half marks, and missing it earned no marks; an independent AI Agent scored all nine reports without knowing the experimenter's scores, with differences of no more than half a point.70% similarUnverifiedAI detectors judge the statistical distribution of text rather than semantic authenticity, and detector accuracy systematically decreases as generative models produce more natural output.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1043791API
curl https://kongchang.com/api/v1/knowledge/claims/1043791MCP
get_claim(id=1043791)