Unverified50% confidenceBenchmarkExact time
On the same machine, the author compared three Gemma variants on 76 synthetic cases: Gemma 4 E4B reached 97%, Gemma 3 4B reached 78% with 5 false positives, and Gemma 4 12B reached 80% with 32 call timeouts.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/18/2026
First Seen
Valid until: 1/16/2027
Sources
Related Entities
Related Claims
UnverifiedThe author empirically compared Gemma 3 1B and 4B: 1B, in the rename scenario, wrote "the app will resend the shelved event" and told the sender to make the change, and both errors were caught by the checks and fell back to the rule answer; 4B took about 16 seconds on a laptop GPU and accurately stated the customer, amount and specific field.69% similarUnverifiedOn the held-out compound recipe test set, the LoRA fine-tuned model caught 66%, while the base Qwen3.5-4B model caught 78%.69% similarUnverifiedOn the BFCL tool-calling test, SmolLM3-3B scores 0.945, slightly above Gemma 4 E2B's 0.927, but the difference is not significant; Qwen3-4B (0.940), Qwen3-1.7B (0.935), Qwen2.5-1.5B (0.910), and Granite 3.3 2B (0.920) are all at the same level as Gemma.67% similarUnverifiedAfter changing the header identification to have Gemma 4 E2B fill only the three required gaps of date, quantity, and product, all 12 header sets were correct with 0 errors, but the author cautions that the same batch of data was also used for debugging, so 12/12 should be regarded as an optimistic estimate.67% similarUnverifiedOn the GSM8K mathematical reasoning test, Qwen3-4B scores 0.915 and Gemma 4 E2B scores 0.911, a difference of +0.4 percentage points with a confidence interval crossing zero, making it the only model on par with Gemma.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1064186API
curl https://kongchang.com/api/v1/knowledge/claims/1064186MCP
get_claim(id=1064186)