Gemini 4 Day-One Hands-On: Best for Documents, Don't Switch for Coding Yet

Google's last Pro-tier flagship model launch was in February of this year. For the following 236 days, it didn't release another flagship. The Gemini 3.5 Pro teased in May was supposed to launch in June — June came and went, then July, then August. By late September, Google simply announced that 3.5 Pro wouldn't ship at all, and went straight to Gemini 4. Gemini 4 made its first appearance on September 30, but only for a select group of cybersecurity organizations; two weeks later, on October 13, it finally opened up to everyone.

After such a long wait, is it any good? A Bilibili creator brought in two other top contenders to "interview" alongside it. This head-to-head review uses a hiring process to string together three conclusions: for reading documents and doing analysis, it's currently number one; for writing code, don't switch yet; the price is half off now, and doubles on December 1.

Introducing the Three "Candidates"

The three contenders first state their background and expected salary. GPT-6 Astra comes from OpenAI, focused on large-scale engineering, with a salary five times that of Gemini 4. Claude Opus 5.5 comes from Anthropic, ranked first in overall scores, and positions itself as "I'm slow, but I do detailed work," with a salary twice that of Gemini 4.

On Google's side, the trial period is half price, lasting through the end of November, then doubling in December. Its resume also lists 19 achievements and 13 first-place finishes — but the resume was written by itself, so the creator decided to do a background check.

Background Check: Is There Padding on the Resume?

The third step is verifying the resume. A third party re-tested it, and most scores matched up. Three things didn't: one place relaxed the exam rules, one place omitted a stronger competitor, and one place swapped in a different set of test questions. On announcement day, Bloomberg reported that people inside Google said it was gaming the benchmarks, which Google denied. There's currently no evidence of resume fraud, but it's very good at "switching exam rooms."

Now look at the third party's own test results. For reading financial reports, reviewing contracts, and doing analysis for you, it ranks first, less than two points above Opus. Its hallucination rate is 15%, versus Astra's 51%. In overall intelligence, it and Astra both score 53, with Opus at 58. But when it comes to the engineering role, its scores drop off: it ranks fourth for writing code in the command line, behind Claude's two models and Astra; and when asked to operate a computer like a human, its success rate is under 5%, second-to-last among 8 models. In one sentence: as an office worker it's an honors student, but in the server room it's still catching up.

What Is the "Hallucination Rate"

The "hallucination rate" mentioned in the article refers to the proportion of responses in which a model fabricates information that sounds plausible but isn't true. This is one of the core metrics for measuring the reliability of large models: when generating fluent, coherent text, a model may state non-existent data, citations, APIs, or facts with complete confidence. Evaluations typically include questions where the model "should admit it doesn't know," testing how it handles uncertain questions.

Gemini 4's 15% hallucination rate means that when it encounters a question it can't answer, it more often chooses to honestly admit it doesn't know rather than force a fabrication, making it more valuable in low-tolerance scenarios like legal and finance; Astra's 51%, by contrast, means it answers confidently even when uncertain, and if it errs, the consequences can be severe.

Day One Live: Examiners Take Turns Setting Questions

With the background check done, it's time for the trial period. On day one live, the examiners take turns giving it problems. The first examiner has it directly modify the low-level code of a 36-year-old Super Nintendo game, and it produces a new level. The work it brings itself is even more outrageous: after watching a video of a person throwing a beanbag, it teaches a robot to hit 7 in a row — though this is Google's own demo.

Next comes same-question comparison, where it's both faster and cheaper. On the lava lamp task, it takes under 2 minutes and 18 cents, while Opus takes 9 and a half minutes and $1.27. On the pixel city task, the results from the same prompt are about the same, with bills of $1.20 versus $8.20. On the long task, run to completion, it takes 38 minutes, Astra 85 minutes, and Fable 130 minutes.

But two examiners paused the frame to look closely. On the ship-in-a-bottle task, it costs 19 cents versus Opus's $12, but Opus's bottle has reflections in the glass and moving water. On the gummy bear task, Opus's can be squeezed and bounces back, while its is a statue — and this time it's actually more expensive: $22 versus $7.20. Cheap doesn't hold every time.

"Same-Question Comparison" and the Logic of Cost Accounting

The article repeatedly uses the same problem to have different models answer, then compares time and cost. This is the increasingly common "task-level cost" approach in model evaluation. Traditional evaluations only look at the token unit price of a single API call, but in actual use, a model may output lengthy reasoning processes and repeatedly self-check, causing total consumption to far exceed unit-price estimates. That's why the article stresses "don't just look at unit price, look at how much it costs to finish one thing."

For example, on the lava lamp task, Gemini 4 finishes in under 2 minutes for 18 cents, while Opus takes 9 and a half minutes and $1.27. The gap mainly comes from the difference in computation driven by speed. But on the gummy bear task it's actually more expensive, showing that the cost advantage isn't constant across different task types.

Three Strict Examiners: Coding Is Still the Weak Spot

Programming blogger Theo tested it with his own codebase. More than half of its judgments about the code were wrong — among the 7 models he tested, only it got more than half wrong. Noah assigned 20 programming tasks, and half the people disappeared on three of them — and they happened to be the three simplest. Glazer posed a set theory problem, and it thought for ten minutes, then stopped on its own; only when switched to a goal-mode tool would it keep grinding. So part of the mid-way stopping is due to default settings.

Why does this happen? An extra test with 100 questions. Astra answered 63 correctly and 19 incorrectly, saying, "I was so confident when I wrote those 19." Gemini 4 answered 50 correctly and only 8 incorrectly, with the remaining 42 being "don't know." It doesn't fabricate questions it can't answer — in places like legal and finance, where saying one wrong thing can cost money, this is an advantage; in places where the job needs to get done, it becomes leaving halfway through. In fact, it's the same thing.

The Difference Between Goal Mode and Default Mode

"Goal mode" in the article refers to a mode in which a model, inside a programming tool, is set to pursue a specific objective (such as passing all tests or completing a specified function) and keeps trying, unlike the default "chat a bit and stop." Gemini 4 stops by default on the set theory problem, but after switching to goal mode it's willing to keep grinding, showing that part of its programming weakness isn't a lack of ability, but rather that the default configuration is too conservative and tends to clock out early when things get hard.

This phenomenon reveals a common problem: when evaluating models, if you don't distinguish between default behavior and tool-enforced modes, it's easy to mistake a configuration problem for a capability problem. For developers, actively setting goal mode or explicitly asking it to "finish before stopping" may significantly improve task completion rates.

Price: Cheap Unit Cost, But Don't Just Look at Unit Cost

First, the barrier to entry. To chat, you need a Google Pro membership, about $20 a month; to use Google's own coding tool for writing code, you need Ultra, starting at $100. This was the most heavily criticized point on launch day.

Now the unit price. It's currently half of Opus and one-fifth of Astra. But it's talkative: on the same test set, it writes more than twice as many words per question as Astra. So per question it comes to $1.99, while Astra is $3.26 — and after it doubles on February 1 to $3.98, it will be overtaken.

Fixing real code is even more direct: its score is 14 points lower than Opus, and it costs more than twice as much per task as Opus. Don't look at unit price; look at how much it costs to finish one thing.

Final Hiring Conclusion

The results are in. For the document role, hire it: contracts, financial reports, and long reports go to it, with human review wherever it doesn't know. For writing code, don't replace anyone yet; keep Opus and Astra. For prototyping and demos, hand it over while it's half price — the trial period lasts until November 30.

If you're only chatting or writing, don't rush to buy a membership for it; there's still no systematic evaluation of its Chinese-language performance. All the numbers in this video come from its first day on the job, and conclusions may change in two weeks.

Finally, about the name. Google has never explained why it's called Argon. In Chinese, Argon is 氩, and the purple neon sign at the beginning is its color; argon gas glows with exactly this purple light when electrified. In 1894, chemists named it using Greek, meaning "lazy," because it doesn't react with anything. You've all seen how it performed on its first day at work, but being non-reactive has its benefits too — the original U.S. Declaration of Independence is sealed in argon gas. For handling documents, it's a professional.

A Fun Fact About Argon and Document Preservation

The article mentions at the end that the original U.S. Declaration of Independence is sealed in argon gas, which is a real artifact-preservation technique. Argon is an inert gas with extremely low chemical reactivity, reacting with almost nothing, so it's used to fill sealed containers to block out oxygen and moisture and prevent paper from oxidizing, acidifying, or molding. The National Archives' "Charters of Freedom" display cases are indeed filled with argon, combined with constant temperature and humidity, to protect the originals of the Declaration of Independence, the Constitution, and the Bill of Rights.

Google naming the model Argon both matches its purple neon sign (argon gas discharges purple light) and hints at its "inert, non-reactive" characteristics — in document-processing scenarios, this conservative "don't fabricate, don't tamper" quality actually becomes a reliability advantage.

One-line Summary

Gemini 4 ranks first for document analysis but fourth for coding, with half-price trial through end of November.

Paragraph Summary

After 236 days, Google launched Gemini 4, and a Bilibili creator compared it head-to-head with GPT-6 Astra and Claude Opus 5.5. The results show that Gemini 4 ranks first in document analysis and financial-report reading tasks, with a hallucination rate of only 15% versus Astra's 51%, and it doesn't fabricate when it doesn't know, making it suitable for legal and financial scenarios. But its coding ability is clearly weak: it ranks only fourth among 8 models, gets more than half of code judgments wrong, and stops midway on simple tasks. On price, its unit cost is half of Opus, but because it outputs more words, cost per question reaches $1.99, and after doubling on December 1 it will lose its advantage. The half-price trial lasts until November 30; Pro membership is about $20/month, and Ultra starts at $100. The conclusion is: hire it for documents, don't replace coders yet, and use it for prototypes and demos while it's half price.