Unverified50% confidenceBenchmarkExact time
In the author's manual verification of 40 Mainz photos, when identifying seats Gemma 4 E4B found 2 true seats with 1 false positive, while the map (using the criterion of a bench within 15 meters of the camera) found only 1 with 4 false positives.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn identifying steps and shade, Gemma performed poorly: E4B falsely reported steps in 16 photos containing no steps, and falsely reported shade in 26 photos containing no shade, so the route planner uses only the seating answer.61% similarUnverifiedWhen Gemma wrote entire messages directly, it dropped all numbers in 4 out of 4 tests and completely reversed the meaning twice.59% similarUnverifiedGemma 4 26B A4B scored a high 96.7% on spatial localization (grounding), but slipped to 76.0% on step-level execution, indicating that open-source models know where objects are but still fall short in simulating the results of continuous actions.58% similarUnverifiedGemma 4 26B terminated with a runtime error before producing a score, because the scoring code assumed at least one sentence would be returned in the correct structured format.56% similarUnverifiedClaude Haiku 4.5 answered 'not found' in 21 of its 24 errors on two-step questions, while GPT-5.4 mini copied the literal wording of the referral as the answer in 20 of its 27 errors, and fabricated real-sounding names while citing actual documents in another 7.55% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1003964API
curl https://kongchang.com/api/v1/knowledge/claims/1003964MCP
get_claim(id=1003964)