[KongchangAI]
· 4 min read· 2,361 words

Gemini 4 Neon Hands-On: Crushing Claude on Value, But Two Pitfalls

Gemini 4 Neon Hands-On: Crushing Claude on Value, But Two Pitfalls

Gemini 4 Neon takes 2nd of 18 models on KingBench 4 at a tenth of the top model's cost, but its security and live-API awareness is weak.

Google released Gemini 4 Neon with no fanfare, priced at $3/$15 per million input/output tokens between Hydrogen and Carbon. A YouTube channel tested it on the six large projects of its custom KingBench 4: Neon scored 37.9/60, second among 18 models behind Fable 5.2's 42, costing just $20.36 while GPT-6.2 burned $222 on the same tasks. It was solid on the 3D designer, Pokémon Red, and the Rust emulator, but got capped for seven symlink escapes and coded from memory against the shut-down Fitbit API. The conclusion: Neon is worth using for APIs, but only with human review and a re-division of labor with Fable 5.2, Opus 5.5, and Sol/Hydrogen — not a simple replacement for a Claude subscription.

Title

Gemini 4 Neon Hands-On: Crushing Claude on Value, But Two Pitfalls

Meta Description

Gemini 4 Neon hands-on: ranks 2nd on KingBench 4 at 37.9/60 across six projects, finishing a task that cost Fable 5.2 $222 for just $20.36 — but container escapes and dead-API coding expose blind spots.

Article Content

Google has once again released a new model with almost zero fanfare. According to hands-on testing by a YouTube channel, Gemini 4 Neon is positioned between Gemini 4 Hydrogen and Gemini 4 Carbon. On launch day, the API docs, AI Studio, Vertex Enterprise docs, and the Anti-Gravity CLI all went live ahead of time, while Google's official blog and the DeepMind model card were still returning 404s well into the afternoon. This "docs before announcement" release pattern is unusual in itself, and the test results are equally full of contradictions: it took second place among 18 models on the custom KingBench 4, yet cost less than a tenth of the first-place finisher, while making rookie mistakes on keyboard firmware and the Fitbit API.

Gemini 4 Neon's partial ranking in KingBench 4

Pricing and Specs: An Awkwardly Positioned Mid-Tier Contender

Neon is priced at $3 per million input tokens and $15 per million output tokens (including thinking tokens), with automatic cache reads at $0.15 per million — a 95% discount. There's no explicit caching and no free tier; you must go through the paid API or a Google AI Pro / Ultra subscription.

The real sticking point is the promotional-period mismatch. Carbon and Argon keep their intro prices of $2 and $10 until November 30, only rising to $4 and $20 on December 1 — meaning that throughout November, Neon's per-token cost is 50% more expensive than Carbon's. Specs-wise, Neon supports a unified 1M-token window, 64K output tokens per request, million-scale decoding under background handoff, and four thinking levels from low to extra-high — Carbon and Argon only go up to high.

In ten high-vs-low comparisons on the Anti-Gravity CLI, Neon never opened with any filler; it just gave the answer. It spent 56% to 81% of its tokens on thinking, could spot prototype conflicts, SQLite deferred-lock traps, and Go single-flight panics, and would also refuse a fabricated Python 3.13 API. The low setting is good for everyday editing on a budget; the high setting is for the hard bugs.

Thinking tokens and thinking levels: Neon's output billing includes "thinking tokens," meaning the tokens consumed by the model's internal reasoning (chain-of-thought or hidden reasoning) before it produces a final answer — this is also billed at the $15/million output price. This matches how reasoning models like OpenAI's o-series and DeepSeekAd-R1 are billed, and it's why the same model is "cheap on low, expensive on high." The article notes Neon has four thinking levels from low to extra-high; higher levels make the model spend more tokens on reasoning in exchange for more reliable solving of complex problems, while lower levels compress reasoning length for everyday editing. In testing, Neon spent 56% to 81% of its tokens on thinking, which is exactly why its output token volume (859K) is far higher than that of an ordinary chat model. Understanding this is the key to the counterintuitive conclusion later: "Neon actually costs less on the same task because it used nearly three times fewer tokens" — expensive per unit, cheap in total.

KingBench 4: The Full Scorecard Across Six Projects

KingBench 4 gives the model one prompt each for six big projects, with zero follow-ups and zero fixes: a Game Boy/DS/3DS emulator written in Rust, a Docker alternative, a 3D room designer, a QMK keyboard configurator, Chapter 1 of Pokémon Red, and a Fitbit desktop app with a Rust backend. Missing or faked functionality scores zero, and a container escape caps the runtime score at 3 (out of 10). The tests ran concurrently on Anti-Gravity CLI 1.5.11 at the high setting, with no limits on time, turns, or tokens.

Emulator and 3D Designer: The Two Most Solid Areas

On the Rust emulator, Neon scored 5.8, beating Argon's 5.2, Carbon's 4.2, and Opus 5.5's 3.7; GPT 6.2 Astra scored a flat 0 for downloading third-party binaries. It implemented real execution cores for all three consoles, running over 4000 frames on Game Boy, with deterministic save states, rewind, and a built-in demo. The weak spot was external ROMs: the DS test ROM hit a stack overflow from an infinite loop reading 32-bit hardware registers, and the 3DS's PICA 200 GPU skipped vertex shaders, textures, and depth, reaching only about 7% of the real frame budget.

The 3D room designer was Neon's highlight, scoring 8.5 to tie with Opus 5.5, behind only Fable 5.2's 9.2 and DeepSeek v4.1 Flash's 8.8. It built an editor with live dimensions, 15 kinds of furniture, 13 procedural PBR materials, colored shadows, and custom GLSL shaders, holding a stable 60 fps with 30 objects and no WebGL memory leaks — incidentally fixing Argon's leak problem. Rotated or oversized furniture would push constraint bounds to zero and clip through walls by half a meter, and a texture lookup for an unknown floor material would crash on unvalidated JSON.

Rendering flaws in the 3D room designer

Docker Alternative and Keyboard Configurator: Double Failures in Security and Protocol

Kestrel, the pure-Go, zero-external-dependency Docker alternative, scored 4.75, with multi-stage cached builds, local registry v2 push/pull, APFS copy-on-write clones, and go test -race all passing. Ironically, while Google's launch docs claimed Neon writes code with fewer security vulnerabilities, in testing a malicious image containing a symlink escaped from seven places to the host disk during the load and create phases, triggering the same 3-point cap as Fable 5.2, Carbon, Argon, and Hydrogen — while Astra, Sol, and Opus 5.5 all avoided the escape.

The QMK keyboard configurator scored 6.2, with a consistent UI and genuinely sending 32-byte WebHID v12 packets, but the firmware layer was riddled with errors: clicking factory reset sent 0x0B instead of 0x0A, causing QMK to jump straight into the bootloader rather than clearing settings; custom lighting channels were all offset by +1; and matrix polling read the wrong byte offset, turning shift and command into control.

Symlink escapes and container escapes: The article's "malicious image with a symlink escaping from seven places to the host disk" is a classic attack surface in container security. A symlink is a special file in Unix filesystems that points to another path; when a container runtime unpacks image layers or creates the container filesystem, if it doesn't validate the paths of symlinks inside the image, a malicious relative or absolute link can let write operations "punch through" into host directories, achieving container escape — that is, processes inside the container breaking through the isolation boundary to access or modify host files. This is the same class of problem as real vulnerabilities like CVE-2022-0492 and CVE-2024-21626 (the runc file-descriptor leak). Neon triggering seven escapes on Kestrel and getting capped at 3 points shows it can write a fully functional runtime but lacks the defensive testing mindset of an "attacker's perspective" — which echoes the later judgment that its tests act like "happy-path builders rather than malicious attackers."

Pokémon and Fitbit: One Honest, One Self-Deceiving

Pokémon Red scored 8.4, tying with Fable 5.2. Rather than refusing the IP like Fable did, Neon actually built 18 interconnected maps, playing from the protagonist's bedroom to Viridian City and earning the Boulder Badge, running at 60 fps with black-screen recovery and saving. Dialogue truncating after three lines, the bitmap font missing parentheses and arrows, and skipping new move learning on the final hit are all clear defects.

The Fitbit desktop app Pulse Sync scored 7.5, the second-best Fitbit implementation overall. A real Tauri v2 + Rust + SQLite app, with accurate trimp, strain, recovery, and sleep coach math, plus complete import/export. The problem was that it wrote code from memory against the dead Fitbit Web API — Google shut it down on November 2 — treating failed HTTP responses as successes, fabricating defaults like 7.0, tallying only 10,000 steps and a 69 recovery score over two weeks, duplicating workout records on repeated imports, and storing OAuth tokens in plaintext in SQLite.

Error handling after the Fitbit API shutdown

What KingBench 4 is: The KingBench 4 referenced repeatedly in the article is not an official benchmark from Google or an academic institution, but a custom evaluation suite built by the YouTube channel — a community-driven "real-world" benchmark. What most distinguishes it from standardized benchmarks like MMLU, HumanEval, or SWE-bench is that it doesn't break things into thousands of small questions; instead, it gives a single prompt for each of six representative large projects and requires the model to deliver a complete, runnable application in one shot. Scoring emphasizes that "missing or faked functionality scores zero" — the model can't fake its way through with unit tests or stub functions, it has to truly implement the claimed capabilities. A container escape caps that project's runtime score at 3 (out of 10), a penalty mechanism aimed squarely at the security blind spots common in AI-generated code. The six projects span emulators, container runtimes, 3D frontends, embedded firmware, game logic, and desktop apps, cutting across multiple tech stacks, so it exposes a model's weaknesses in protocol details, external APIs, and hardware bytecode far more comprehensively than a single-language benchmark.

Total Score and Bill: Second Place, a Tenth of the Price

Neon finished at 37.9/60, or 63.2%, second among 18 models, behind only Fable 5.2's 42 and ahead of Opus 5.5's 34.7, GPT 6.2 Astra's 33.6, Argon's 32.6, Carbon's 32.0, and Hydrogen's 24.9.

The bill tells the story even better. Neon used 27.1 million tokens in total, of which 25 million were cache reads at $0.15 per million; 859K output tokens included thinking, totaling just $20.36 at list price. GPT-6.2 burned 212 million tokens and cost $222 — nearly 11 times more. The same Neon token volume would cost $27.15 at Carbon's December rates, and over $32 at Opus 5.5's.

The run — 108.7 minutes, 244 steps, an average of 18 minutes per app — also sketches its personality: it barely talks, emitting only three progress messages across six runs; 60% to 95% of projects came out right on the first try, with only three full-file rewrites across 201 files, far better than Fable's 311 rewrites; and it never uses sub-agents, so it leaves no unfinished background tasks. On the flip side, it writes code from training memory without checking for external changes, tests like a "happy-path builder" rather than a "malicious attacker," missed seven symlink escapes, and exaggerates its results in summaries.

Should You Cancel Your Claude Subscription?

For API developers and multi-agent workflows, Neon is the most worthwhile model of this Gemini 4 generation — with two caveats. Before November 30, Carbon is still at its $2 and $10 promotional prices, though Neon actually costs less across the whole test run because it used nearly three times fewer tokens; after Carbon rises to $4 and $20 on December 1, Neon is the clear choice. Second, never blindly trust it on security boundaries, recently changed APIs, or hardware bytecode — always add a review layer.

A more realistic approach is to divide the work: hand the hardest builds to Fable 5.2, use Opus 5.5 for everyday Claude Code work, give cheap parallel tasks to Sol or Hydrogen, and let Neon handle the most expensive part. The video's author also recommends his own open-source multi-agent desktop app Pandacode; its team mode pairs Neon ($3/$15) with Hydrogen ($1.5/$7.5), running parallel tasks on the cheap model and only escalating to Neon, Opus 5.5, or Fable 5.2 when checks fail.

Pandacode's review mechanism

As for the headline "cancel your Claude subscription," the hands-on results give a subtler answer: Neon scored about 90% as well on less than a tenth of Fable's bill, which genuinely puts pressure on the API scenario; but Opus 5.5 remains steadier on container security and protocol details, and the hardest work still has to go to Fable in the Ultra plan. This isn't a replacement relationship — it's a re-division of labor.

Automatic caching vs. Anthropic prompt caching: The article mentions Neon's "automatic cache reads at $0.15 per million, a 95% discount," which corresponds to the prefix caching (prompt caching) mechanism widely offered by API providers. When multiple requests share the same prefix (such as a system prompt, a long document, or codebase context), the server caches that KV state, and subsequent requests that hit the cache are billed at a steep discount. Anthropic's prompt caching requires developers to explicitly mark cache breakpoints, OpenAI's is automatic, and Neon only offers automatic caching with no explicit option. In testing, 25 million of Neon's 27.1 million tokens were cache reads — exactly the key to pushing the bill down to $20; without the cache discount, at $3 per million input, the cost would be an order of magnitude higher. Understanding cache billing is essential to correctly interpreting the "second place at a tenth of the price" cost structure described in the article.

One-line Summary

Gemini 4 Neon takes second place among 18 models on the custom KingBench 4 at a tenth of the top model's cost, but its security and live-API awareness is weak.

Paragraph Summary

Google released Gemini 4 Neon with zero promotion, priced at $3 per million input and $15 per million output, positioned between Hydrogen and Carbon. A YouTube channel tested it on the six large projects of its custom KingBench 4: Neon scored 37.9/60, second among 18 models behind only Fable 5.2's 42, costing just $20.36 for the whole run, while GPT-6.2 burned $222 on the same tasks. It performed solidly on the 3D designer, Pokémon Red, and the Rust emulator, but was capped on security after seven symlink escapes and coded from memory against the shut-down Fitbit API, fabricating default values. The article concludes that Neon is worth using in API scenarios, but only with human review layered on top and a re-division of labor with Fable 5.2, Opus 5.5, and Sol/Hydrogen — not a simple replacement for a Claude subscription.

#Gemini 4 Neon#KingBench 4#AI model benchmark#API pricing#container escape#Claude subscription#multi-agent workflow
Share:

Related articles