Unverified50% confidenceBenchmarkExact time
In a comparative test on 29 real blunders, the bare Gemma model mentioned illegal moves not on the board 7 out of 29 times.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/17/2026
First Seen
Valid until: 1/15/2027
Sources
Related Entities
Related Claims
UnverifiedIn a comparative test on 29 real blunders, the bare Gemma model correctly named Stockfish's top alternative move 0 out of 29 times.75% similarUnverifiedIn 11 of the 12 models, between 87.5% and 100% of capitulation cases stopped at the exact wrong number provided by the user, meaning capitulation typically involves switching sides entirely.61% similarUnverifiedModels that only read the call transcripts passed 29 of the 31 contradictory cases.61% similarUnverifiedIn a standalone test of the "view incomplete" prompt on 100 questions, the wrong-answer rate was 16% without the prompt and dropped to 5% with it.61% similarUnverifiedIn validation testing, the model refused all 27 requests for prohibited capabilities and all 23 requests for unsupported operations.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1053950API
curl https://kongchang.com/api/v1/knowledge/claims/1053950MCP
get_claim(id=1053950)