Gemma 4 31B
A 31-billion-parameter open-source large language model in Google's Gemma 4 series. When quantized, it requires approximately 16–20 GB of memory and can run on consumer-grade Apple Silicon devices.
Timeline (last 90 days)
On the ring_push task, four models (Claude Haiku 4.5, Gemma 4 31B, GPT-5.4 mini, Gemini 3.7 Flash) completed the required changes in both tasks, yet lost old behaviors not restated in the prompt, passing fail-to-pass but failing pass-to-pass, and this failure reproduced stably across eight runs.
Among the 100 labels in the classification task, fraud and normal each accounted for 50, the majority-class baseline is 50%, and Gemma 4 31B's 55% is statistically insufficient to prove it surpasses the baseline.
On the same 100 samples, the GraphSAGE graph neural network baseline correctly classified 67 samples, outperforming Gemma 4 31B's 55.
On 100 samples, Gemma 4 31B achieved 55% accuracy in transaction classification and 53% accuracy in neighborhood suspicious transaction identification (a proxy metric).
A developer built a benchmark task in the Kaggle Benchmarking Challenge, using the IEEE-CIS fraud detection dataset to test Gemma 4 31B (google/gemma-4-31b) on its ability to judge whether a transaction is suspicious based solely on textualized graph structure information.
The eight models tested include GPT-6 Luna, Claude Haiku 5.5, Gemini 3.5 Flash-Lite, Qwen 3 Next 80B, Gemma 4 31B, GPT-6 Sol, Claude Sonnet 5, and Gemini 3.7 Flash, with each model running 107 investigations per track.
Claude Haiku 4.5, Gemma 4 31B, and Qwen3-235B verbatim copied the coordinates provided by users, and two other models added a prohibited "points" field on GPS checkpoint requests.
Larger, newer models do not necessarily perform better: gpt-oss-120b (0.740) scored lower than gpt-oss-20b (0.761); DeepSeek-R1, costing $1.94, scored lower than Gemma 4 31B, which cost only 13 cents.
Google may want to run a mid-sized model similar to Gemma 4 31B on phones, running relatively efficiently on approximately 32 GB of memory.
In memory recall tests, the 12B model answered 82 out of 100 questions correctly and the 31B model answered 98 out of 100, with neither model hallucinating answers.
4 more timeline events
All Facts (14)
On the ring_push task, four models (Claude Haiku 4.5, Gemma 4 31B, GPT-5.4 mini, Gemini 3.7 Flash) completed the required changes in both tasks, yet lost old behaviors not restated in the prompt, passing fail-to-pass but failing pass-to-pass, and this failure reproduced stably across eight runs.
50%UnverifiedAmong the 100 labels in the classification task, fraud and normal each accounted for 50, the majority-class baseline is 50%, and Gemma 4 31B's 55% is statistically insufficient to prove it surpasses the baseline.
50%UnverifiedOn the same 100 samples, the GraphSAGE graph neural network baseline correctly classified 67 samples, outperforming Gemma 4 31B's 55.
50%UnverifiedOn 100 samples, Gemma 4 31B achieved 55% accuracy in transaction classification and 53% accuracy in neighborhood suspicious transaction identification (a proxy metric).
50%UnverifiedThe eight models tested include GPT-6 Luna, Claude Haiku 5.5, Gemini 3.5 Flash-Lite, Qwen 3 Next 80B, Gemma 4 31B, GPT-6 Sol, Claude Sonnet 5, and Gemini 3.7 Flash, with each model running 107 investigations per track.
50%UnverifiedClaude Haiku 4.5, Gemma 4 31B, and Qwen3-235B verbatim copied the coordinates provided by users, and two other models added a prohibited "points" field on GPS checkpoint requests.
50%UnverifiedLarger, newer models do not necessarily perform better: gpt-oss-120b (0.740) scored lower than gpt-oss-20b (0.761); DeepSeek-R1, costing $1.94, scored lower than Gemma 4 31B, which cost only 13 cents.
50%UnverifiedIn memory recall tests, the 12B model answered 82 out of 100 questions correctly and the 31B model answered 98 out of 100, with neither model hallucinating answers.
50%UnverifiedThe experimental setup used a single NVIDIA H100 serving via vLLM, with test models Gemma 4 12B and Gemma 4 31B, and input data consisting of 50 million tokens of real public text.
50%UnverifiedOpus 5.5 at the Max tier requires $14.65 to achieve the same score as GPT-6.1 Sol, meaning 6.1 Sol is approximately 73 times cheaper.
50%Unverifiedclaude-code-local claims to support models including Qwen 3.5 122B (approximately 65 tokens/second), Llama 3.3 70B, and Gemma 4 31B.
50%UnverifiedThe Gemma 4 31B version, while maintaining a relatively small parameter count, attempts to strike a balance between conversational quality and breadth of knowledge.
50%UnverifiedA developer built a benchmark task in the Kaggle Benchmarking Challenge, using the IEEE-CIS fraud detection dataset to test Gemma 4 31B (google/gemma-4-31b) on its ability to judge whether a transaction is suspicious based solely on textualized graph structure information.
50%UnverifiedGoogle may want to run a mid-sized model similar to Gemma 4 31B on phones, running relatively efficiently on approximately 32 GB of memory.
50%