Why Gemini 3.5 Pro Is Stuck in Development Hell — and Why Google Is Betting Everything on Flash

Flagship model delays, key talent departures, and Flash iterating at breakneck speed — Google has been sending some unusual signals lately. Has it fallen behind in the AI race, or is its understanding of that race fundamentally changing?

The Anomaly: Pro Stalls, Flash Sprints

Gemini 3.5 Pro — once eagerly anticipated — was said to be launching "next month" a few months back, yet it still hasn't officially shipped. Meanwhile, Google has been iterating Flash at a furious pace, releasing 3.5 Flash, 3.6 Flash, 3.7 Flash, and now 3.8 Flash in the span of just six weeks.

What makes this even more striking is that these model roadmap shifts coincided with notable turbulence in Google's AI leadership. Demis Hassabis stepped back from day-to-day management of DeepMind to become Chairman and Alphabet's Chief Scientist. Jeff Dean, Sanjay Ghemawat, and other heavyweights shifted roles. There were even reports that AlphaFold lead researcher John Jumper might be heading to Anthropic.

On the surface, this looks like Google's flagship models losing momentum. But if you put the intelligence scores, speed, task costs, and Google's own TPU infrastructure together, something more interesting emerges: Google may be redefining what this AI war is actually supposed to be about.

From Competing on Intelligence to Competing on Cost

The old game was about who had the smartest model — highest benchmark scores, strongest reasoning, most leaderboard wins. Google now seems to be playing a different game entirely: how much effective intelligence can you produce per million dollars spent, or per megawatt-hour consumed?

Here's a telling set of numbers. Gemini 3.8 Flash scores around 59 on the AI intelligence composite — not the top mark. The highest-tier models sit around 66, with several mainstream competitors clustered between 60 and 63. So on raw intelligence alone, 3.8 Flash isn't the clear leader.

But shift your attention to speed, and the picture changes completely. Gemini 3.8 Flash outputs approximately 302 tokens per second — first place by a wide margin. The second-fastest sits around 182 tokens per second. It may not be the smartest, but it generates text at a strikingly fast pace.

The truly decisive metric, however, is Cost Per Task — the weighted average cost to complete an equivalent intelligent task. Gemini 3.8 Flash comes in at around $0.58, while some competing models cost $2.34 or even $3.69 for comparable work.

What does this mean in practice? A gap of 59 versus 63 or 66 intelligence points doesn't sound dramatic. But if the cheaper model costs a fraction as much and runs significantly faster, real-world deployments will amplify that gap enormously. A lab runs a benchmark once. A production AI service runs continuously — thousands of calls per second, around the clock.

A note on benchmarks: Benchmarks are standardized evaluation suites used to measure model capability — common examples include MMLU (multi-domain knowledge), HumanEval (code generation), and MATH (mathematical reasoning). For the past two years, benchmark rankings served as the primary marketing metric in AI, with "top of the leaderboard" virtually synonymous with technical leadership. But benchmarks have well-documented limitations: scores can be inflated, intentionally or not; static test environments don't capture real-world long-tail requirements; and critically, they measure accuracy while completely ignoring latency and cost. The "AI intelligence score" referenced in this article is essentially a weighted aggregate across multiple benchmarks. By presenting it alongside speed and cost, Google is deliberately nudging the industry toward evaluating models by "how much intelligence per dollar" rather than "whose score is higher."

The Primary Work Model: Built for the Agent Era

This is why Google has positioned 3.8 Flash as a critical workhorse — not a chat interface enhancement, but a model purpose-built for coding, agentic tasks, multi-step reasoning, and tool use. Because these are the workloads most likely to generate massive call volumes going forward.

Imagine an agent completing a task on your behalf: it reads files, searches the web, invokes tools, writes code, runs tests, detects errors, re-reasons, revises, and runs again. A single end-to-end task might require dozens or even hundreds of model calls. At that scale, what enterprises actually care about isn't "is your model three points smarter than mine" — it's "how much does it cost to get the same job done."

Currently, 3.8 Flash is priced at approximately $0.75 per million input tokens and $3.75 per million output tokens — consistent with 3.7 Flash's launch pricing. Earlier, 3.6 Flash reduced average output tokens by roughly 17% compared to 3.5, with certain code tasks seeing reductions as high as 65%.

There's an easy-to-miss optimization happening here: Google isn't just making Flash faster. It's making Flash output fewer — but more effective — tokens, accomplishing the same tasks with less computation. Across the journey from 3.5 to 3.8, the core metric being optimized is increasingly important: intelligence cost per unit.

A note on agents: An agent (or AI agent) is an AI system capable of autonomously planning, invoking tools, and executing multi-step tasks — fundamentally different from traditional single-turn question answering. In a single-turn exchange, the user asks, the model answers, one inference happens. In agent mode, the model iterates through "think → act → observe" cycles, each triggering new model calls. An automated coding agent fixing a bug might sequentially invoke code reading, web search, code generation, unit test execution, and error analysis tools — generating dozens of independent API requests for a single task. This means agent workloads are far more compute-intensive than chat, with single-task costs potentially tens of times higher. As a result, inference latency and per-token pricing become dramatically amplified in agent deployments, making "fast and cheap" models far more attractive in practice than "slightly smarter" alternatives.

Billions of Users: An Advantage That's Also a Cost Problem

Why does Google care so intensely about this metric? Because its greatest strength — billions of existing users — is also one of its biggest AI-era cost pressures.

A traditional Google Search query followed a simple, relatively cheap compute path: parse the query, retrieve web pages, rank results, insert ads. The cost was bounded; the commercial return was high. AI-powered search is fundamentally different. Ask "help me plan a seven-day trip to Japan" and the system might crawl dozens of web pages, organize hotel and itinerary options, call mapping services, compare transportation modes, and run multiple reasoning passes. What the user experienced as a single question may have triggered dozens of model calls on the backend.

The math becomes uncomfortable: the larger Google's user base, the better the AI, the more it gets used — and the bigger the compute bill. This dynamic differs from OpenAI or Anthropic. When OpenAI adds a paying user, it gains a subscription. When Microsoft sells a Copilot seat, it books new revenue. Google's situation is different: these users already exist. As Google embeds AI into Search, Android, YouTube, and its broader ecosystem, much of the resulting call volume comes from traffic that was already there — and in many cases, already free.

Top-tier model APIs now process hundreds of millions to over a billion calls per day. Every $0.01 reduction in per-call cost can save Google hundreds of millions annually — a far better return than simply scaling capital expenditure.

There's a deeper pressure too: AI may disrupt Google's most profitable machine — search advertising. If users accept AI-generated answers directly and stop clicking traditional search results, where do the ads go? Why would advertisers pay as much for that traffic? Google is simultaneously absorbing the new compute costs of AI while trying to prevent AI from cannibalizing its existing profit engine. That's the hardest equation it faces.

The TPU Factor: How Much Intelligence Per Megawatt?

This is where TPUs become critical. Evaluating AI chips today can't stop at how fast a single chip runs. The real question is more practical: how much AI compute can you produce per megawatt of electricity?

According to estimates from Epoch AI, when normalized to H100-equivalent compute, Google's TPU solution delivers approximately 2,039 H100-equivalent units per megawatt. By comparison, NVIDIA's B200/B300 full-rack solution comes in around 1,175 H100-equivalent units per megawatt. Note that these figures measure whole-datacenter system efficiency — not a direct chip-to-chip comparison. The same analysis estimates that Google, as a single organization, holds roughly one-quarter of global AI compute capacity.

What Google is actually optimizing for is this: for every dollar spent and every kilowatt-hour consumed, how many tokens, tasks, and units of effective intelligence can be produced? When you serve billions of users, the most powerful competitive advantage may not be "we have the world's smartest model" — it may be "we can shave just a little more cost off every inference call." That sounds incremental, but multiplied across billions of daily calls, it becomes a staggering number.

A note on TPUs: The Tensor Processing Unit is Google's custom-designed AI chip, now in its sixth generation (Trillium) since the first iteration in 2016. Unlike NVIDIA GPUs — which use a general-purpose parallel computing architecture — TPUs were purpose-built from day one for matrix multiplication and neural network inference, giving them higher efficiency for Transformer-based models under specific workloads. More importantly, Google co-optimizes TPUs with its proprietary inter-chip interconnect (ICI), datacenter cooling, and power management as a unified system — rather than simply stacking third-party chips. This vertical integration strategy allows Google to achieve efficiency advantages at the system level rather than the chip level. The Epoch AI figures cited (2,039 vs. 1,175 H100-equivalent per megawatt) measure the combined efficiency of the full datacenter solution, not bare chip performance. For a company with massive internal call volumes, this system-level efficiency advantage translates directly into lower marginal cost per inference.

From Arms Race to Industrial Race

With that context, Gemini 3.5 Pro's delay and 3.8 Flash's rapid iteration start to look very different. Google clearly can't abandon Pro — it represents the intelligence ceiling. If Google consistently loses the most complex reasoning, research, and coding tasks to OpenAI and Anthropic over the long term, it will fall behind. Flagship models still matter enormously.

But the strongest model isn't necessarily the one that gets called the most — or the one that makes the most money. This will be especially true in the Agent era, where the vast majority of real-world workloads prioritize "good enough, cheap, and fast." Whoever executes on those three dimensions wins the bulk of enterprise business.

This shift isn't unique to Google. Microsoft, OpenAI, Anthropic, and many AI labs globally are placing increasing emphasis on cost-efficient models, model tiering, and inference economics. The AI war of the past few years resembled an arms race — biggest model, most parameters, highest score. The next phase looks more like an industrial race: who can deliver the highest-value output at the lowest cost per task.

Google is the company least able to ignore this problem. Many competitors are still figuring out how to find users. Google's actual concern is: if billions of users genuinely start using AI at high frequency, how do we pay for it? Its challenge isn't a lack of technology, compute, or users — quite the opposite. It has too many users. When the cost per inference drops even slightly, multiplied by the scale of Google's call volume, the impact flows directly to the bottom line.