Gemini 4 Carbon Hands-On: The Strongest AI Model? Compared with Claude Opus 5.5

Google has suddenly released its new-generation frontier model Gemini 4 Carbon, officially claiming it ranks first on the Artificial Analysis leaderboard and sets new records in cybersecurity and long-horizon tasks. Combining a Bilibili creator's early hands-on test with official data, let's see where it's truly strong and whether it deserves the title of "strongest ever."

Benchmarks: Topping the Charts, but Beware the History of Inflated Scores

According to Google's announcement, Carbon tops the Artificial Analysis Intelligence Index 5.0 leaderboard while setting new highs in cybersecurity and long-horizon decision-making tasks. Most notably, it scored 71% on the CW Bench test, which requires the model to effectively diagnose and fix the root causes of security flaws, demonstrating strong real-world security capability.

Google's internal tests also show that Carbon's success rate in generating secure code is more than double that of the previous generation, Argon. The company even asserts that its prompt injection attack success rate is "completely zero" and that its hallucination rate has dropped significantly. However, the creator reminds us that Argon also had impressive benchmark scores at launch, yet its real-world experience fell noticeably short of Opus—so Carbon's scores should be viewed with cautious optimism.

Artificial Analysis Intelligence Index is a composite model capability index maintained by the independent evaluation organization Artificial Analysis, typically integrating reasoning, coding, math, knowledge Q&A, and other dimensions, weighted into a single score. It is characterized by fast updates and coverage of mainstream closed-source and open-source models, so vendors often cite it at launch. However, its weighting design may not reflect real usability in specific scenarios, and models may have been optimized for public benchmarks.

CW Bench is a benchmark focused on real-world cybersecurity capability. Its questions mostly come from real vulnerability scenarios, requiring the model to locate the root cause of flaws and provide actionable remediation plans rather than merely identifying vulnerability patterns. It is closer to a security engineer's daily workflow than traditional code completion tests, so a score of 71% is regarded as a high real-world threshold.

Long-Horizon Task Breakthrough: Topping Vending Bench

Carbon took first place in the Vending Bench test, which is its most underrated highlight. This benchmark evaluates whether an AI can autonomously run a simulated vending machine business, requiring it to manage inventory, pricing, and finances and make hundreds of interlocking decisions while maintaining long-term strategic coherence. Most models fall short in such long-cycle business simulations; Carbon's top ranking means the ability of agents to maintain autonomous operation of business-like projects has reached a new level.

Long-horizon tasks refer to task types that require a model to maintain goal consistency, remember context, and dynamically revise strategies across dozens or even hundreds of interaction steps. Unlike single-turn Q&A, the core difficulty of long-horizon tasks lies in error accumulation: an early wrong decision propagates to all subsequent steps, placing extremely high demands on the model's planning, state tracking, and self-correction capabilities. Vending Bench is a representative of such benchmarks, coupling variables such as inventory, pricing, and cash flow in a vending machine operation simulation to test whether an agent can maintain business logic consistency over a long cycle rather than merely completing single-step operations.

Hands-On Experience: Stunning Front-End Design, Coding Slightly Behind Opus

The creator's hands-on test shows that Carbon has made significant progress in 3D asset generation and front-end animations. For example, it generated a luxury kitchen scene purely from code, with glass placement, material textures, and meat rack details all quite refined—lighting and shadows have flaws but are overall stunning. A scroll-triggered animation homepage also earned praise for its visual aesthetics and attention to detail.

However, in pure web development, Carbon still lags slightly behind Claude Opus 5.5, while on most coding tasks it is on par with Opus. Given that its price is far lower than Opus, the creator says he is willing to use it as his daily primary model.

3D asset generation refers to a model directly outputting renderable three-dimensional scenes or objects, usually with code (such as Three.js or WebGL shaders) or structured geometric descriptions as an intermediate representation. It is harder than generating 2D images because it must simultaneously satisfy geometric topology, material textures, and lighting consistency, and the code must be executable. The detailed performance of the "luxury kitchen scene" in the evaluation reflects the model's joint control over spatial layout and material parameters, not just visual aesthetics.

Pricing and Rollout Rhythm: Low Price, High Performance

Carbon continues Argon's launch pricing: $2 per million input tokens, $10 for output, with a 95% discount on cached input. Combined with its performance, its cost-effectiveness is extremely prominent. In terms of rollout, Google first opened it to cybersecurity defenders through the FairWing program, then gradually pushed it to all users over the following weeks; currently, ordinary users cannot yet access it.

Google's Next Move: A Stronger Gemini Is Already on the Way

DeepMind pretraining researcher Zachary made a meaningful remark in a discussion on the day Carbon was released: "Gemini isn't online yet," hinting that a model far more capable than Carbon is in development, speculated by outsiders to be Gemini 4 Ultra. At the same time, a training checkpoint codenamed Hydrogen was also mentioned. Google is clearly continuing its periodic table naming system and maintaining an aggressive iteration pace.

Other Highlights: Fable 5.5 Leak, Claude Updates, and Hardware News

According to leaks, a model suspected to be Fable 5.5, Obsidian 3, appeared in the Frontier Code 2.0 benchmark, scoring 46.8% at high reasoning intensity—only slightly ahead of Opus 5.5's 45.5%—and dropping to 45.5% at maximum reasoning intensity, a tie, while costing about $47 per inference, 3.6 times that of Opus 5.5 at equivalent intensity. If true, the improvement is disappointing, and the impressive performance of early checkpoints has not carried over.

On the Claude side, users can type UltraThink in the input box to directly adjust reasoning intensity; Claude Code has launched dynamic workflows, supporting calls to multiple sub-agents via editable JavaScript files, executed serially or in parallel and savable as slash commands for reuse. In addition, reports suggest there is a hidden points system behind Claude account bans.

Google's Q3 earnings show AI's significant contribution to growth: Alphabet revenue grew 25% year over year, reaching an annualized scale of $500 billion; Google Cloud revenue grew 103%, and first-party model APIs process nearly 320 data points per minute, up 40% quarter over quarter. Because demand exceeds infrastructure capacity, Google plans to significantly increase third-party compute in Q4, raising capital expenditure from $195 billion to $205 billion.

Other developments include: Nvidia released Nemotron 3.5 Transcribe, a 30-billion-parameter mixture-of-experts speech recognition model that activates only 4 billion parameters during inference, supports 40 languages and speaker identification, and achieves an average word error rate of 5.33% on the Hugging Face OpenASR English leaderboard; the Motion Bricks system generates 15,000 frames of motion imagery per second with only 2 ms latency on an RTX 5090; and Apple launched its first touchscreen MacBook Pro, equipped with a ProLED XDR touch display, which may change AI development and creative workflows.

Mixture of Experts (MoE) is a model architecture that splits a large model into multiple "expert" subnetworks, activating only a subset of parameters during each inference, thereby controlling computational cost despite a large total parameter count. Nemotron 3.5 Transcribe's "30 billion total parameters, only 4 billion activated" follows this approach, enabling a speech recognition model to reduce inference latency and compute overhead while maintaining multilingual and speaker identification capabilities.

Conclusion: The Title of Strongest Remains to Be Verified, but the Direction Is Right

Carbon demonstrates in both benchmark scores and hands-on tests Google's determination to close its gaps, and its low-price strategy gives it the potential to become a primary model. But "strongest ever" still needs large-scale validation by real users, and DeepMind's hint suggests Carbon may only be the beginning. For developers, a cheaper and more versatile Gemini is clearly worth looking forward to.

One-line Summary

Google releases Gemini 4 Carbon, topping benchmarks with high performance at low price, but the "strongest ever" title still awaits real-world validation.

Paragraph Summary

Google's new-generation frontier model Gemini 4 Carbon has been released. Google claims it tops the Artificial Analysis Intelligence Index 5.0, scores 71% on the CW Bench cybersecurity test, ranks first on Vending Bench for long-horizon tasks, and asserts a zero prompt injection success rate and reduced hallucination. A Bilibili creator's hands-on test shows significant progress in 3D asset generation and front-end animations, with pure web development slightly behind Claude Opus 5.5, but at a fraction of the price, offering outstanding cost-effectiveness. The article also cautions about Argon's precedent of inflated benchmark scores and rounds up the Fable 5.5 leak, Claude feature updates, Google's Q3 earnings, and simultaneous Nvidia and Apple developments, noting that DeepMind hints a stronger Gemini is still on the way.