Unverified50% confidenceBenchmarkExact time
In tests on 42 candidate answers about Romanian legal deadlines, the author's own AI safety check system accepted all 28 wrong answers (14 wrong party and 14 wrong action), while rejecting 6 of the 14 correct answers.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedWhen the open-source small model gemma2:2b was used as the checker, it accepted 8 of the 28 wrong answers (6 wrong party and 2 wrong action), while rejecting 10 of the 14 correct answers.75% similarUnverifiedIn validation testing, the model refused all 27 requests for prohibited capabilities and all 23 requests for unsupported operations.67% similarUnverifiedOf the 962 responses, 51 failed, distributed as 26 truncations, 13 boundary errors, 12 paraphrases, and 0 corrections.67% similarUnverifiedAmong the five evaluators the authors tested, if the answer key had minor errors, the score was only 0.21 — lower than giving no key at all (0.46) and lower than giving a completely irrelevant key (0.50); while a human-verified model-generated answer scored 0.61, close to the 0.69 of a human-written one.67% similarUnverifiedOf the 69 artificially injected document problems (wrong check digits, unregistered numbers, deregistered issuers, other people's numbers, wrong tax amounts), all 69 were routed to manual review, and 95.7% were caught by the corresponding validations.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1045676API
curl https://kongchang.com/api/v1/knowledge/claims/1045676MCP
get_claim(id=1045676)