Unverified50% confidenceFactExact time
The scoring method was a weighted detection score: clearly describing a bug earned full marks, merely mentioning it earned half marks, and missing it earned no marks; an independent AI Agent scored all nine reports without knowing the experimenter's scores, with differences of no more than half a point.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe core of evaluation is scoring, typically expressing the quality of results using a continuous value from 0 to 1.71% similarUnverifiedAI judges show outcome bias when scoring: when only the run outcome is changed (while the Agent's decision process is the same), the AI judge gives a lower PROCESS score in the version where the outcome is wrong.70% similarUnverifiedagent-ops-check scores on a range from 0 to 100, deducting 10 points for high-risk findings, 4 points for medium-risk, and 1 point for low-risk, and the repository's own bad example scores 27.70% similarUnverifiedVendor-reported coding scores are typically measured on carefully curated subsets, and results reproduced by independent organizations on the full test set are often lower.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1043903API
curl https://kongchang.com/api/v1/knowledge/claims/1043903MCP
get_claim(id=1043903)