Gemini 4 First Test: Strong Value for Money, but Generation Quality Still Trails Competitors

Google has finally released Gemini 4, and since it's been a while since the last major version number update, the first batch of hands-on experiences has drawn considerable attention. An early hands-on video from YouTube offers first impressions across several dimensions—leaderboard ranking, 3D generation, game controllability, and value for money—and the conclusions aren't one-sided: Gemini 4 shows strengths on the price-efficiency curve, but lags behind competitors like Sonnet 5.5, GPT 6.1 Sol, and Opus 5.5 in multiple generation quality comparisons.

Leaderboard Position: Mid-Tier, but Sitting on the Value Frontier

According to Agent Arena data cited in the video, Gemini 4 ranks eighth, slightly above Sol 5-6 but below the latest models such as 6 Sol, Fable 5, and Opus 5. More noteworthy is its performance on the Pareto frontier—the Gemini series has consistently held an edge in pricing efficiency, and Gemini 4 does indeed make it onto the Pareto frontier, though in the middle zone. This means that across thousands of user-submitted tasks, its overall ranking sits in the middle of the pack.

Pricing is a clear advantage for Gemini 4. The video creator estimates its cost at roughly two-thirds of 6 Sol, and about 70% cheaper than Opus 5.5. Models like DeepSeek^Ad are far cheaper still, but within its price band, Gemini 4's pricing strategy remains competitive.

Pareto Frontier and Pricing Efficiency

The Pareto frontier is originally a concept from economics and multi-objective optimization: the set of solutions that form the frontier when no objective can be improved without worsening another. In the context of model evaluation, the two axes are typically "cost" and "task completion quality/win rate," with each model plotted on the chart—a model on the frontier means it's hard to find a higher-quality alternative at the same price, or a cheaper alternative at the same quality.

So making it onto the Pareto frontier doesn't mean a top overall ranking; it means the model has no obvious disadvantage in the price-performance trade-off. Gemini 4 sits in the middle of the frontier, which reads as: it's neither the cheapest nor the strongest, but it delivers acceptable performance at its mid-range price point, making it a viable candidate when budget is constrained. If a model is simultaneously dominated by another model on both dimensions, it's called "dominated."

Rankings on crowdsourced evaluation platforms like Agent Arena are based on battles or ratings from a large number of user-submitted tasks. Unlike static benchmarks, they're closer to real-world usage distributions, but they're also affected by the distribution of submitted task types, and rankings fluctuate considerably—best treated as a reference rather than a definitive verdict.

3D Generation Hands-On: Golden Gate Bridge Holds Up, Later Scenes Fall Behind

The video creator ran a quick comparison across a series of 3D generation prompts. On the classic "Golden Gate Bridge" prompt, Gemini 4 performed notably well—coherence and 3D texture were both solid, the water texture was impressive, and the UI was thoughtfully done. The creator considers this on par with or even slightly better than Sonnet 5.5, though GPT 6.1 Sol still comes out ahead.

However, starting from the second scene, the gap widens. In "island shelter"-style prompts, Gemini 4's output showed problems like trees falling at random angles and misplaced elements; by contrast, Sonnet 5.5's characters and floating islands were rich in detail and coherent, with clearly superior visual appeal. The creator stated outright that this one was "definitely not the best."

The "Tower of Babel" scene further exposed weaknesses: characters floating, stuck motionless, stiffly arranged, and water handling problems. In the comparison with GPT 6.1 Sol, the latter's characters had clearer design intent and behavioral purpose, with an overall level of completion a tier higher. The same goes for the White House scene—Gemini 4 wasn't bad, but had flaws like anomalous black blocks on the roof; Sonnet 5.5 delivered refined design sensibility, walking characters, and beautiful floral details, which the creator described as "on a completely different level."

It's worth noting that the creator emphasized he doesn't care about 3D generation per se—these examples are meant to observe a more fundamental capability: stability of instruction following. The same model varies noticeably across different prompts—excellent in some scenes, clearly unstable in others—and this instability is the real warning sign. If detail control breaks down even in generation tasks, then on tasks requiring precise thinking, such as database schema design, users may need to guide the model more explicitly.

Instruction Following and Controllability

Instruction following refers to how well a model adheres to explicit constraints in a prompt (counts, layout, spatial relationships, style, prohibitions) when generating content; controllability is broader—whether users can reliably steer output toward the intended direction via prompts, rather than getting random drift on every generation.

In 3D or scene generation tasks, these two capabilities manifest as: whether objects are placed as described, whether characters behave sensibly, and whether physical relationships (water surfaces, support, occlusion) are self-consistent. Large quality swings for the same prompt across different scenes usually indicate the model lacks stable internal representations—this isn't merely an aesthetic difference. Aesthetic issues can be fixed in post; structural failures are hard to repair.

This is also why the reviewer stresses that "3D generation is just a vibe check": generation tasks are merely probes for observing underlying capabilities. If a model frequently destabilizes on spatial and numerical constraints, then on tasks requiring precise structural adherence—database schema design, configuration generation, code refactoring—users often need to break requirements down more finely and provide more explicit constraints and examples to get reliable output.

Game Generation: Insufficient Controllability

The Gemini series previously had a good reputation for game generation, but Gemini 4's performance in this hands-on test was disappointing. When the creator tried the generated mini-games, the control system was "pretty bad," to the point of making the gameplay hard to understand, and characters would get stuck. Compared with games generated by GPT 6.1 Sol, whose controls also aren't exactly excellent but are at least playable and don't have characters running around everywhere, the contrast is clear. The creator acknowledged that a single game isn't enough to fully judge game generation capability, but at least this time there was no obvious advantage for Gemini 4 in this dimension.

Model Version Numbers and Capability Expectations

A major version number (like Gemini 3 to Gemini 4) usually implies an overall update to the base model, training data, or post-training strategy, but it doesn't guarantee that every capability dimension improves in step. Release cadence, compute allocation, and product positioning all influence trade-offs across dimensions: some versions emphasize reasoning and cost efficiency, others emphasize multimodality or long context.

So "a new version is necessarily stronger across the board" is a common misjudgment. The rational approach is small-scale A/B testing on specific task sets: pick the 5–10 task types your team handles most frequently, run the same batch of prompts through both old and new models, and compare success rate, rework rate, and cost per task. Cost-sensitive teams should also factor price into the metrics, comparing "cost to complete one task" rather than raw token price—this is how the Pareto frontier perspective translates into practical model selection. Brand and version number should only serve as initial screening criteria, not a substitute for hands-on testing.

How to Interpret This First Test

The value of these first impressions lies in revealing the outline of Gemini 4's positioning: it continues the Gemini family's traditional strength in pricing efficiency, successfully enters the Pareto frontier, and suits cost-sensitive scenarios with moderate task difficulty. But in generation quality and stability, it has yet to catch up with the tier of Sonnet 5.5 and GPT 6.1 Sol, especially with fluctuating cross-scene consistency.

The creator's reminder is worth remembering: 3D generation is just a "vibe check"; what really matters are instruction-following stability, creative expressiveness, and detail controllability—capabilities that transfer to real workflows. For teams considering switching models, Gemini 4 is worth trying firsthand, but decisions shouldn't be made on brand or version number alone.