Gemini 4 Carbon Released: Tops Third-Party Leaderboard, But Ordinary Developers Still Left Out

Google released Gemini 4 Carbon on October 28, calling it its most powerful frontier model to date. Just 15 days after the previous generation, Gemini 4 Argon, was made available, Google explained the unusually short iteration cycle by saying "researchers and models worked hand in hand to speed up iteration." On the new version of the intelligence index from third-party evaluator Artificial Analysis, Carbon ranks first with 59 points, ahead of Claude Opus 5.5 at 58 and GPT-6 Astra at 57.

However, this "world No. 1" currently has little to do with most developers. Carbon is available only to cyber defenders in the Fairwind program, and ordinary developers will have to wait. This reveals a notable trend: the launch strategy for frontier models is shifting from "casting a wide net" to "targeted enablement," with security becoming a priority deployment scenario.

Performance Leap: Significant Progress in Coding and Agent Tasks

The areas where Carbon improved most noticeably this time are coding and agent tasks. Terminal Bench rose from 57 to 65, Agent's Last Exam climbed from 69 to 77, and on Web Dev Arena it jumped by 89 points at once to rank fourth.

These figures echo the screenshot information that "long context is its strength"—in scenarios requiring multi-step reasoning and tool calls, Carbon demonstrates stronger task-completion capabilities. However, on the coding agent index, Carbon's 59 still trails the 62 of Claude Code paired with Opus 5.5, indicating that the Claude family retains an advantage in the pure coding agent track.

Long Context: The Core Weapon That Widens the Gap

Long-context processing is the most prominent technical label for Carbon this time. In Graphworks' test, where the model must locate information within contexts ranging from 250,000 to 1 million tokens, Carbon achieved 96.2% accuracy, while GPT-6 Astra managed only 71.8%.

A gap of this magnitude means that when handling large codebases, long-document analysis, or complex multi-turn conversations, Carbon's usability will be noticeably higher than that of competitors. For enterprise scenarios requiring "needle-in-a-haystack" information retrieval, this capability has direct productivity value.

Long Context Window refers to the total number of tokens a model can receive in a single inference pass. A token is the basic unit of text a model processes, roughly equivalent to 0.75 English words or half a Chinese character. The larger the context window, the more information the model can "see" at once, making it better suited for handling large codebases, lengthy contracts, or long-term memory tasks spanning multiple conversation turns. The industry commonly uses the "Needle-in-a-Haystack" test to evaluate long-context capability: a key piece of information is buried in hundreds of thousands of tokens of irrelevant text, and the model must retrieve it accurately. Carbon's coverage of the 250,000 to 1 million token range in the Graphworks test means it can theoretically read in about 750,000 English words of material at once—directly meaningful for enterprise-level document analysis and code review.

Cybersecurity Becomes the Main Thread: From Finding Vulnerabilities to Writing Patches

Cybersecurity is the main task for this generation of models. SEC Bench Pro, which tests vulnerability discovery, scored 88.2%; Exploit Gym, which verifies whether vulnerabilities can actually be exploited, rose from 42% to 54.8%; and CWE Bench, which tests patch writing, reached 71%.

These three metrics form a complete defensive chain: discovering vulnerabilities, verifying exploitability, and generating remediation patches. Google's decision to limit Carbon's initial release to cyber defenders is precisely based on this capability positioning. The jump in Exploit Gym from 42% to 54.8% deserves attention—verifying vulnerability exploitability is one of the most time-consuming parts of security work, and improved model capability could significantly compress response times.

CWE Bench: CWE stands for "Common Weakness Enumeration," a software security defect classification system maintained by MITRE that covers hundreds of weakness types, including buffer overflows, injection attacks, and missing permission checks. CWE Bench uses known weaknesses in real codebases as samples, requiring the model to generate a remediation patch that eliminates the weakness and to verify whether the patch passes functional and security tests. Exploit Gym, meanwhile, evaluates whether a model can turn a discovered vulnerability into an actually exploitable attack path—that is, verify the vulnerability's "exploitability." The two correspond respectively to the "remediation" and "verification" links in the defensive chain, and together with SEC Bench Pro's vulnerability discovery capability they form an end-to-end security evaluation framework.

Weaknesses Are Also Obvious: Rising Hallucination Rate and Information Opacity

Carbon does not come without costs. Independent testing shows that its "swap-out rate" (i.e., hallucination or erroneous generation rate) rose from 4% to 11%, meaning that while performance improved, reliability declined somewhat. For security scenarios, this number needs to be treated with caution.

Another signal worth noting is that Google did not release a technical report or disclose the model's parameters. This differs from Google's past information-disclosure habits for frontier models, making it difficult for outsiders to understand the architectural source of Carbon's capabilities or to independently reproduce or assess its risk boundaries.

Hallucination refers to a model generating content that appears plausible but is inconsistent with the facts or fabricated out of thin air. In security scenarios, the harm of hallucination is amplified: the model may invent non-existent vulnerability IDs, provide incorrect patch logic, or claim that a piece of code is secure when it actually contains flaws. The "swap-out rate" rising from 4% to 11% in this article means that roughly 1 in every 9 outputs may contain erroneous content. For defenders relying on models for vulnerability assessment and patch generation, this level requires retaining a human review step; model output can only serve as auxiliary reference rather than a final conclusion.

Pricing and Accessibility: Restrained Pricing, But the Barrier Remains

On pricing, Carbon has not raised prices. The introductory price is $2 per million input tokens and $10 per million output tokens, with cached input charged at only 5% of the original price. This pricing strategy is competitive among competitors, but accessibility is the key issue.

Currently Carbon is available only to cyber defenders in the Fairwind program. For the vast majority of developers and enterprises, no matter how strong the model is, if it cannot be called, it cannot be converted into actual value. Google's choice to conduct closed testing first in the security domain may be out of consideration for controlling capability risks, but it also leaves the "world No. 1" leaderboard result on paper for now.

Carbon's Positioning from a Multi-Source Cross-Perspective

Combining information from Bilibili creators and multiple third-party review channels, the consensus advantages of Carbon center on two capabilities: long context and cybersecurity, and its top ranking on the AI index is a fact confirmed by multiple sources. Divergence appears in the coding agent track: some analyses believe Carbon's overall coding capability is already close to the Claude family, but the gap of 59 versus 62 on the Agent index is still regarded by most evaluators as a real weakness. In addition, a single source pointed out that the rise in the "swap-out rate" may be related to targeted optimization for security scenarios, but this claim has not yet been corroborated by other sources.

For ordinary developers, Carbon's release is more like a signal than a tool: the iteration speed of frontier models is accelerating, the capability ceiling is rising, but the access threshold is also being redrawn in a scenario-based way. Waiting may be the only certain choice at the moment.

One-line Summary

Gemini 4 Carbon tops the intelligence index with standout long-context and security capabilities, but is limited to cyber defenders.

Paragraph Summary

Google released Gemini 4 Carbon on October 28, just 15 days after the previous generation. The model ranks first on the Artificial Analysis intelligence index with 59 points, ahead of Claude Opus 5.5 and GPT-6 Astra. Its core strengths center on long context and cybersecurity: in Graphworks testing, it achieved 96.2% accuracy in locating information within 250,000 to 1 million token contexts, far exceeding competitors' 71.8%; its security capabilities cover the full chain of vulnerability discovery, exploitability verification, and patch generation. Its coding agent index score of 59 still trails the 62 of Claude Code with Opus 5.5. On weaknesses, the hallucination rate rose from 4% to 11%, and Google did not release a technical report or parameter information. Currently Carbon is available only to cyber defenders in the Fairwind program, and ordinary developers cannot call it yet.