[KongchangAI]
· 4 min read· 2,296 words

Gemini 4 Hydrogen Released: Hands-On Testing of the Fastest, Cheapest Flash Model

Gemini 4 Hydrogen Released: Hands-On Testing of the Fastest, Cheapest Flash Model

Gemini 4 Hydrogen redraws Flash model boundaries with low cost and high reliability amid a day of multi-threaded AI news.

The article centers on Google's Gemini 4 Hydrogen, using hands-on testing data to outline its speed, cost, and capability boundaries: 2.5x faster than Gemini 3.8 Flash, about 32% of Sonnet 5.5's per-task cost, an 8% hallucination rate, with reliability as the biggest highlight—though coding still trails on Terminal Bench 4.0. It also covers Anthropic's IPO roadshow, Nvidia's leadership shakeup and Blackwell forensics controversy, OpenAI's 77% ExploitGym score, and other product updates.

Google pushed the meaning of the "Flash" suffix one step further in a single day. Gemini 4 Hydrogen has officially launched, positioned as the fastest and cheapest Flash model to date, reportedly 2.5 times faster than Gemini 3.8 Flash. A Bilibili creator simultaneously released their own hands-on benchmark data in a video, making the model's capability boundaries concrete.

Speed and Price: The Cost-Performance Curve of Flash Models

The most intuitive change comes from cost. According to the Artificial Analysis independent benchmark cited by the creator, Gemini 4 Hydrogen scores 54 on the Intelligence Index at the X-High reasoning tier—only 1 point below Claude Sonnet 5.5 and GPT-6.1 Sol at their highest reasoning tiers. For a model positioned as Flash, that gap is almost unnaturally small.

Gemini 4 Hydrogen's performance on visual reasoning and reliability

The price structure tells a different story: the cost to complete a single task is about 32% of Sonnet 5.5's, but still roughly 2.5 times more expensive than GPT-6.1 Sol. In other words, Hydrogen isn't the "cheapest" in absolute terms—it's cheap relative to its own predecessor and peer large models, not relative to OpenAI's product line. At the High reasoning tier, it can also match Argon's intelligence score at about 19% of Argon's cost.

The model's API pricing is listed at $1.50 per million input tokens and $7.50 per million output tokens.

Artificial Analysis Intelligence Index is a widely cited third-party aggregate metric for evaluating large models, maintained by the independent analysis firm Artificial Analysis. It normalizes and weights scores from multiple academic and practical benchmarks—including MMLU-Pro, GPQA Diamond, Humanity's Last Exam, LiveCodeBench, and SciCode—into a composite 0–100 score for cross-vendor comparison of "general intelligence." Its distinguishing feature is that it records both the reasoning tier (thinking budget) and cost per task, making it suitable for assessing "how much intelligence you get per dollar." Note, however, that it is ultimately a weighted average of benchmarks, with limited coverage of long-horizon agentic tasks, tool-calling stability, and other dimensions, so it may diverge from real-world production performance.

Four Reasoning Tiers and "Near-Carbon" Performance

Gemini 4 Hydrogen introduces four thinking levels: Low, Medium, High, and X-High, letting users trade off reasoning depth, speed, and cost. More notable is how it collaborates with Gemini 4 Carbon—through Antigravity's new orchestrator, Carbon handles planning while Hydrogen executes coding and testing tasks, and Google claims this multimodal combination can approach Carbon-level performance.

One easily overlooked detail: the previous generation Gemini X 3.0 Flash was described as built on Gemini 3 Pro, whereas Hydrogen is described as adapted from Gemini 4 Carbon. The creator suggests this may hint at a shift in Google's training philosophy—smaller models are more closely related to larger ones—which also explains how a Flash model can perform this well. He even speculates that Google may already have some form of RSI (recursive self-improvement) mechanism driving model iteration. This is personal speculation with no public evidence to support it.

RSI (Recursive Self-Improvement) refers to AI systems participating in improving the next generation of AI systems—for example, automatically generating training data, designing architectures, writing training code, or optimizing evaluation pipelines—thereby creating a cycle of accelerating capability iteration. It is discussed repeatedly because once the loop closes, improvement speed could detach from the bandwidth limits of human engineers. Currently, there is no public evidence that any lab has achieved fully autonomous RSI; existing practice is closer to "AI-assisted R&D," where models handle data cleaning, code generation, and experiment orchestration while humans retain key judgment. Therefore, the creator's claim that "Google already has an RSI mechanism internally" should be regarded only as speculation about iteration speed, not a verifiable conclusion.

Benchmark Results: Reliability Is the Biggest Highlight

On the creator's custom World of AI Benchmark, Hydrogen ranks 7th overall, ahead of Argon and slightly below Claude Sonnet 5.5—with the gap mainly in coding ability.

Reliability data is more telling: it ties for first on tests including GDP and PDF, with a hallucination rate of 8%, compared to 43% for Sonnet 5.5 and 52% for GPT-6.1 Sol on the same evaluation. On Terminal Bench 4.1, it reduced fabricated statements by more than 5x and handles failed tool calls more stably. It debuted at 5th place on Agent Arena, surpassing Fable 5.1 on agentic work tasks.

Its weaknesses are equally clear: on Terminal Bench 4.0, Hydrogen scores 54%, trailing Sonnet's 64% and Sol's 56%. The X-High tier boosts intelligence but nearly doubles output token consumption.

Terminal Bench is a benchmark for evaluating AI agents' ability to complete engineering tasks in real command-line environments. Tasks cover file operations, dependency installation, script debugging, Git operations, and more, requiring models to execute commands over multiple rounds inside a container and self-correct based on output. Scoring is typically based on task success rate, so it reflects coding ability, long-horizon planning, and recovery from failed tool calls simultaneously. Terminal Bench 4.0 and 4.1 have different task sets and difficulty levels, so scores are not directly comparable across versions; in this article, Hydrogen's 54% on 4.0 and its "5x reduction in fabricated statements" on 4.1 are results on two different dimensions, and version differences should be kept in mind when comparing.

Hands-On: Minecraft Clone and Front-End Landing Pages

The creator asked Hydrogen to generate a Minecraft clone, and after 8 minutes of runtime got a complete result with torches, day-night cycles, block breaking, and a crafting system—producing this level of complexity from a Flash model struck him as "a bit crazy."

Front-end development was another surprise. In a comparison test generating an Xbox controller SVG, Hydrogen took about 4 minutes and cost about 42 cents, while Gemini 4 Argon took about 6 minutes and cost about 83 cents—and Hydrogen's output quality was better. Asked to generate multiple landing pages with scroll triggers and different fonts, the model not only completed the layout but injected its own design judgment.

In terms of user experience, Hydrogen performs best in the Antigravity environment with the Swarm feature. A project called Crimson Moth ran for about 45 minutes with Swarm, and its visual quality approached that of Anthropic-family models.

The model is available through the Gemini App, Google Search, and Google AI Studio; AI Pro and Ultra subscribers can also access it via Antigravity, Gemini in Chrome, and Google Sheets.

Anthropic's Roadshow: Willing to Slow Down for Safety

According to the same video report, Anthropic made a rather unusual statement during its IPO roadshow: the company is willing to slow down AI development for the sake of safety, even if it means falling behind competitors like OpenAI. The founders' letter explicitly states a willingness to "proceed more slowly than we could," even if that gives competitors an advantage.

Supporting commitments include: Claude remains ad-free; seven founders pledge to donate 80% of their Anthropic shares; and the company contributes 5% of its equity to the previously discussed public interest fund.

The creator is skeptical—the last time Dario talked about needing to slow down development, Anthropic subsequently released the Claude 5.5 series, which became one of the strongest models at the time. He sees this as more of a pre-IPO PR narrative. On the financial side, the company's quarterly revenue is reported to be in the billions, growing more than 12x year-over-year, with over 11,900 customers spending at least $100,000 annually; at the same time, it recorded a net loss of $23.3 billion, of which $21.9 billion comes from valuation-related accounting adjustments rather than operating expenses. The stock is expected to trade on Nasdaq under the ticker INT.

During the same period, Anthropic also launched Claude Meetings: Claude joins team video meetings as a visible participant, can answer questions in real time, create documents and whiteboards during the meeting, automatically take notes, and generate action items. The feature starts with Google Meet and is available in beta to team and enterprise subscribers.

Anthropic says Claude helped its model run on AMD chips in three days

Another case mentioned in the roadshow: Claude helped one of Anthropic's models run on AMD chips in three days with almost no human intervention—work the company says would previously have required dozens of engineers and months.

Nvidia Leadership Shakeup and the Blackwell Mystery

The most shocking news reported in the video is the departure of Nvidia CEO Jensen Huang. According to reports, the matter involves allegations of personal relationships and potential conflicts of interest, the board came under enormous pressure, and the current CFO has taken over as interim CEO.

Nvidia's leadership change is seen as a major variable for the AI industry

Meanwhile, a newly released forensic analysis claims that four Chinese AI developers—DeepSeekAd, Moonshot AI, MiniMax, and Ant Group—used Nvidia Blackwell hardware when developing their latest models. The researchers' method is worth noting: rather than relying on leaked documents, they analyzed public model weights to look for numerical patterns that could serve as "hardware fingerprints." Six models reportedly exhibit characteristics consistent with Blackwell, and some of these characteristics have been reproduced on real Blackwell GPUs.

It must be emphasized that these are only the researchers' findings, not independently confirmed evidence of violations. Even if the hardware identification is accurate, it does not establish who supplied the chips, where they were run, or whether export controls were violated. There is also currently no evidence linking the report to Huang's departure. But at a time of peak AI demand, Nvidia faces both a leadership vacuum and supply-chain scrutiny, and the spillover effects on the entire AI ecosystem are worth continued observation.

Model weight forensics is a class of hardware-tracing methods that does not rely on internal documents or supply-chain records: researchers analyze publicly available model weights to look for systematic numerical traces left during training—such as activation distributions in specific layers, quantization error patterns, anomalous outliers in attention heads—and compare them with characteristics exhibited by small models trained on known hardware. The advantage of this approach is that the evidence comes from public artifacts and can be reviewed by third parties; the limitation is that the features may be erased or confounded by post-training processing (fine-tuning, quantization, distillation), and it cannot prove the chip supply path, deployment location, or whether export controls were violated. It can therefore only serve as a lead, not stand alone as evidence of a violation.

OpenAI's Cybersecurity Model: 77% on ExploitGym

At the Cyber at the Frontier showcase, OpenAI unveiled an unreleased internal model that scores 77% on the ExploitGym benchmark—which tests whether AI agents can turn known software vulnerabilities into usable attack code. For comparison, Gemini 4 Carbon scores 54.8%, GPT-6 Cyber 52.6%, GPT-6 Astra 42.4%, and Claude Mythos 5.2 39%. That's a 24.4-point improvement over GPT-6 Cyber, released 8 days earlier. On PatchBench, the model scores 79.1%, versus 73.5% for GPT-6 Cyber.

The iteration speed of such specialized models is worth watching: the same capabilities that let defenders find weaknesses faster could also be exploited by attackers. The model currently has no announced release date or detailed system card, and the creator hopes OpenAI will eventually open these capabilities to vetted researchers through Daybreak Blue and Red.

Other Notable Developments

  • Seed Dance 2.5 Flash: ByteDance's faster, cheaper text-to-video model, with visual quality close to Seed Dance 2.5 and roughly twice the generation speed. 720p text-to-video and image-to-video are priced at about 15 cents per second, roughly one-third cheaper than the flagship model. The API is scheduled to launch on November 15 via BytePlus's ModelArk platform.
  • Handoff Squared: The second-generation computer-use model from new company Hark, claimed to be twice as fast as its predecessor and faster than Opus 5.5 at the high reasoning tier. It's said to improve end-to-end browser task performance by about 10%, buy coffee 1.7x faster, book flights 3.8x faster, and shop 2.5x faster; in its first month it handled over 5.6 million browser sessions and over $20 million in transactions.
  • Microsoft Ignite: The conference opens on November 17, with the MAI team bringing new models, demos, and sessions led by model builders.
  • ChatGPT DOTS update: The personal AI agent becomes more conversational and better at following instructions, with a new custom instructions feature that lets you set approval rules such as "always show me the draft before sending an email."

ChatGPT DOTS custom instructions and behavior settings

  • Gemini in the car: The Gemini app is now integrated directly into Apple CarPlay, letting iOS users have free-flowing conversations with Gemini Live in their vehicles.

Looking at the day's news together, Google has moved the boundary between "cheap and fast" and "good enough and reliable" a little further with a Flash model, while Anthropic's IPO narrative, Nvidia's leadership uncertainty, and OpenAI's specialized cybersecurity model point to three different tension lines: business, supply chain, and security.

One-line Summary

Gemini 4 Hydrogen redraws the boundaries of Flash models with low price and high reliability, amid a day of intertwined AI developments.

Paragraph Summary

The article centers on Google's release of Gemini 4 Hydrogen, combining hands-on data from a Bilibili creator to outline the Flash model's speed, cost, and capability boundaries: 2.5x faster than Gemini 3.8 Flash, about 32% of Sonnet 5.5's per-task cost, a hallucination rate of just 8%, with reliability as its biggest highlight—though its coding ability still trails Sonnet and Sol on Terminal Bench 4.0. The article also analyzes the four thinking levels, orchestration with Carbon, and the training-philosophy shift hinted at by closer kinship between small and large models. In addition, it rounds up other same-day developments: Anthropic's IPO roadshow pledge to slow down for safety and the launch of Claude Meetings, Nvidia CEO Jensen Huang's reported departure and the Blackwell hardware forensics controversy, OpenAI's unreleased cybersecurity model scoring 77% on ExploitGym, and product updates from Seed Dance, Handoff Squared, Microsoft Ignite, and more.

#Gemini 4 Hydrogen#Flash model#AI benchmark#Anthropic IPO#Nvidia Blackwell#OpenAI ExploitGym
Share:

Related articles