[KongchangAI]
· 2 min read· 1,417 words

Gemini 4 Neon Hands-On: Crushes Opus at Coding, but the Free Agent Hides Three Major Pitfalls

Gemini 4 Neon Hands-On: Crushes Opus at Coding, but the Free Agent Hides Three Major Pitfalls

Gemini 4 Neon tops coding benchmarks, but hidden reasoning-token billing is steep and free Tasks fail silently.

Google released the Gemini 4 Neon model and the consumer Agent feature Tasks. Neon hit 58.6 on the Coding Agent Index for global #1, 65.4% on Cursor Bench 4.1, cut task cost from $10.04 to $6.44, and is priced at $3 input / $15 output. But it has no free developer API quota, and every hidden reasoning token in Thinking mode bills at the $15 output rate. Neon also aced four real code tests, finding 10 security flaws in 134 seconds for 25 cents. Free Tasks is available in the US, Canada, Australia, India, Japan, and South Korea but blocked in the UK and EEA, with silent Duplex call failures and no completion notifications.

Google released two major updates within the same time window: a new Gemini 4 model codenamed Neon, and Tasks, a personal Agent feature tucked into consumer-grade Gemini. The former shot to global #1 among coding agents on an independent leaderboard, while the latter opened up free to all users in select regions. But one YouTube creator ran four rounds of real code battles and live stress tests to reach a split conclusion—half impressive, half prone to silently failing in production.

One Release, Two Products, and Almost Nobody Noticed How They Interlock

Laying out the timeline, Google's recent model release cadence has become hard to track: Carbon focuses on long-horizon deep research, Hydrogen handles high-speed sub-agent tasks at low cost, and Project Opal graduated directly into Gemini Tasks. Neon sits between Hydrogen and Carbon, priced at $3 input / $15 output, with 95% context caching and four levels of controllable thinking budget, including an "extra high" tier.

What really surprised the community was its performance on independent leaderboards. On Artificial Analysis V9.01, Neon scored 60.1 overall—just one point behind GPT 6.2 Astra, which is priced at $40—while beating both Carbon and Opus 5.5. On the Coding Agent Index, Neon took global #1 with 58.6, scored 65.4% on Cursor Bench 4.1 and 56.3% on Automation Bench, and cut Opus's task cost from $10.04 to $6.44.

A caveat: these numbers come from a single YouTube creator's account, and readers should verify against the official leaderboard pages.

Google quietly edited a sentence in its release blog post

One Quietly Deleted Sentence, and Two Billing Rules That Bite

In the creator's investigation, Google's release blog post initially contained a sentence stating that Neon was trained by scaling reinforcement learning on top of Gemini 4 Hydrogen. Hours later that sentence was deleted, with no changelog explanation. What the base checkpoint actually is remains officially unexplained.

More practical than the training details are two billing rules. First, Neon has no free API quota for developers. Second, every hidden reasoning token in Thinking mode is billed at the $15 output rate—even if you're still carrying a 1M-token context window. That means the real bill for heavy reasoning can far exceed intuitive expectations.

Context Caching is a technique that lets a model reuse already-computed intermediate states for repeated prompt prefixes. When the beginning of a prompt (such as a system prompt, database schema, or code map) stays unchanged across multiple calls, the server can cache that computation, and subsequent calls only pay for the new content. Cache hits are typically billed at a lower input rate, while the uncached portion is still charged at the standard input price. The "95% context caching" mentioned in the article refers to support for an extremely high proportion of cache reuse, while "raising the fixed system schema ratio from 0% to 95%" means stabilizing the prompt prefix to improve hit rates and drive down unit cost. Note that caching only applies to the input side; hidden reasoning tokens generated in thinking mode are still billed in full at the output rate.

Four Rounds of Code Battles: Where Neon Wins, Where Carbon Loses

Synthetic benchmarks lie easily, so the tests went straight to real tasks.

Test 1: Orbital data center. Same blank file, same prompt. Carbon at the high thinking tier timed out past 8 minutes, and only after dropping to medium did it produce clean geometry and orbital physics in about 100 seconds. Neon at the high tier completed a 24,000-token 3D dataset in 167 seconds for $0.37, and it ran clean on the first try—pulsing neural conduits, a responsive thermal-view HUD, sharper volumetric light, zero console errors.

Test 2: Dark glass simulator. Neon finished in 136 seconds for $0.32. This also validated the caching strategy: raising the fixed system schema ratio from 0% to 95% cut cached input cost to $0.15 per million tokens—less than half of Astra's $0.40 cash rate.

Test 3: Playable 3D orbital trench collider. A single HTML file, completed in 287 seconds, with smooth camera roll, thruster particles, and collision gates showing no state corruption under real-time arrow-key input.

Test 4: Security audit. In Anti-Gravity, switching Neon to X-High reasoning and feeding it an exploitable Next.js billing endpoint—containing a timing-leaking bearer check, missing tenant org ID filtering, and an unbounded agent retry loop—Neon in 134 seconds for 25 cents cleanly found 10 vulnerabilities and patched in SHA-256, constant-time comparison, strict tenant isolation, and a three-strikes retry circuit breaker.

Test results revealed

Coding Agent Index and Cursor Bench 4.1 are both evaluation leaderboards aimed at coding agents, but with different emphases. Coding Agent Index typically measures end-to-end completion rate on real repository-level tasks, tool-calling correctness, and cost efficiency; Cursor Bench is based on actual usage scenarios in the Cursor editor, evaluating code completion, multi-file edits, and instruction following. Automation Bench focuses on the success rate of automated workflows (such as form filling and multi-step operations). Scores on these leaderboards are heavily influenced by prompts, tool sets, and runtime environment, and are mostly maintained by third parties or the community—hence the article's reminder to verify against official pages. Understanding the differences among these benchmarks helps judge exactly which capability boundaries Neon's "global #1" covers.

Free Agent Tasks: Usable, but Don't Hand Over the Keys Just Yet

Google officially named this consumer-grade Agent suite Tasks, integrating Gemini Spark and graduating Project Opal. The availability map is quite fragmented: completely free in the US, Canada, Australia, India, Japan, and South Korea; other regions require a Pro or Ultra subscription; the UK and European Economic Area are entirely blocked.

The product form is an extra chat/task toggle in the prompt bar of Gemini on web and mobile. Switching into Tasks generates a background orchestration workspace powered by Gemini 4 Hydrogen, directly connected to Docs, Sheets, Slides, Calendar, Linear, Jira, and Instacart. More practical still are the native SMS and RCS channels, plus the email channel via gemini.google.com—forwarding an email on the go can trigger agent work. The enablement path is buried deep: on mobile, Settings → Preferences → Channels, then verify your phone number under Text with Gemini.

The hidden path to enabling Tasks

The problems show up in real use. The creator listed three scenarios that fail silently:

  • Duplex auto-calling still fails, silently on many accounts, falling back to an ordinary email after 45 minutes instead of actually placing the call.
  • No notification when long tasks complete, with the thread staying silent until you type a single word, "update."
  • Product tone mismatch, naming an ambient autonomous teammate with the cold label "task" makes it feel more like a one-time boarding pass than a persistent personal operator.

Details of the security audit patches

Duplex is Google's conversational technology for voice-call scenarios, enabling AI to place calls in natural speech on the user's behalf to complete tasks like bookings or inquiries. It requires real-time speech synthesis, interruption handling, and telephony network bridging, with far higher latency and stability demands than text interaction. If the Duplex auto-calling feature in Tasks fails silently on some accounts, it's usually related to regional telephony compliance, carrier lines, or account verification status rather than model capability. RCS (Rich Communication Services) is a rich-media messaging protocol that replaces traditional SMS, supporting read receipts, group chats, and multimedia cards. Tasks receiving instructions via native SMS and RCS channels means users can trigger background agent work without opening the app—this is also key to its "ambient" experience, but channel availability depends heavily on carrier and regional support.

Three Deployment Strategies You Can Use Tonight

  1. Allocate thinking budget by blast radius. Use the cheap Hydrogen for sub-agents, Neon High for everyday coding, and reserve X-High only for high-consequence scenarios like security audits.
  2. Put immutable database schemas and code maps at the very top of the prompt. That way Neon's cache prefix can hit, significantly cutting the cost of an expensive call loop.
  3. Don't let context evaporate between tools. Neon for coding, Cloud 5.5 for UI design, Tasks for daily errands—once context is lost between silos, the rhythm breaks. The creator built Exogram.ai as an encrypted personal memory layer for this purpose; that's a single-source product recommendation, so evaluate as needed.

The final verdict can be blunt: hook Tasks up to SMS and email for free daily workflows, keep Hydrogen on worker threads, leave Carbon for deep research, and Neon deserves to be your default engineering workhorse—provided you're prepared for the hidden reasoning-token bill and the silent failure notification mechanism.

#Gemini 4 Neon#Coding Agent Index#Gemini Tasks#AI Agent#Duplex
Share:

Related articles