The AI Token Cost Crisis: Why Tech Giants Are Collectively Cutting AI Budgets

A Budget Reversal Nobody Saw Coming

Silicon Valley is playing out a fascinating drama: CFOs at tech companies are sitting down, doing the math on how much their AI departments are actually burning, and arriving at nearly the same conclusion — they can't afford it.

According to CNBC, Palo Alto Networks CEO Nikesh Arora publicly stated that AI token prices need to fall by as much as 90% before mass adoption becomes truly viable. Meanwhile, Tesla, Uber, Microsoft, and Meta are all pulling back on AI budgets to varying degrees.

What is a token? A token is the basic unit of measurement a large language model uses to process text. Roughly speaking, one token corresponds to about 0.75 words in English; Chinese varies due to character density. When users call APIs like GPT-4 or Claude, charges are based on the total of input tokens plus output tokens. GPT-4o, for example, is priced at approximately $5 per million input tokens and $15 per million output tokens. Reasoning models (such as the o1 series) cost several times more — even ten times more — than standard models because they generate large volumes of intermediate "thinking" steps. This makes tokens not just a technical concept, but the central variable in enterprise AI spending.

What you might not have noticed is that the companies cutting budgets aren't outsiders from traditional industries — they're the tech companies that should be getting the highest return from AI. The irony is hard to miss: if even the core beneficiaries are starting to count pennies, does the industry's economic model have a fundamental problem?

The Twilight of "Token Maximization"

Not long ago, the industry was buzzing with a completely opposite narrative — "token maximization."

NVIDIA CEO Jensen Huang has a widely cited saying: if he pays an engineer $300,000 a year, he expects that engineer to consume $150,000 worth of tokens. If they haven't spent that much, he'd actually question whether the engineer is getting full value from AI. In that framing, more token consumption meant fuller utilization of AI and higher output value.

But when CFOs actually picked up their calculators and multiplied "$150,000 × total number of engineers," the result was a staggering number. What was once celebrated as a symbol of efficiency suddenly became a crushing line item on the financial statements.

This 180-degree shift reflects a classic herd mentality in tech: not long ago, everyone was championing token maximization; now, everyone has pivoted simultaneously to cost control. The tide went out faster than it came in.

Arora's 90% Price-Cut Thesis

Arora's view isn't an isolated one. Global consulting giant Accenture has run into the same wall repeatedly over the past two years while trying to sell AI to clients: enterprise executives consistently say they can't see a real return on their AI investments.

The logic is straightforward: when you can't see returns, it usually means costs are too high relative to benefits. To make the investment feel worthwhile, token costs have to come down. Arora's specific roadmap: token costs need to drop 20% in the next 12 months, then compress significantly year after year, reaching a cumulative reduction of roughly 90%.

For comparison, OpenAI CEO Sam Altman also told CNBC that the latest models from frontier labs have improved token efficiency in agentic coding by 54%. Altman noted "we probably need to go at it one more time." But even a 54% improvement, while substantial, still falls well short of the 90% reduction Arora is calling for.

High AI token costs have become a primary barrier to large-scale enterprise adoption, keeping many willing customers on the sidelines.

Why Agents Are Such a Money Pit

Behind the runaway token costs lies an architectural problem that's easy to overlook.

Looking at how large language model usage has evolved: it started simply — send a request, get a response. Then came prompt engineering, then context engineering, stuffing RAG knowledge base information into the context window.

RAG and the token bloat problem: Retrieval-Augmented Generation (RAG) is one of the most popular enterprise AI architectures today — retrieving relevant document chunks from a vector database and feeding them into the model's context window to compensate for the model's knowledge cutoff. But the larger the context window, the more tokens each call consumes. As GPT-4's context window expanded from 8K to 128K to even million-token scales, engineers tend to "stuff it as full as possible," ignoring that this directly scales up API costs proportionally. The emerging discipline of context engineering exists precisely for this reason — its central question is: how do you precisely control the scope of context while maintaining model performance, so you're not wasting expensive token quota on irrelevant information?

What truly caused costs to explode is the rise of reasoning models and agents:

• Reasoning tokens: A reasoning model generates an internal chain of thought before delivering a final answer — OpenAI's o1/o3 series and DeepSeek-R1 are prime examples. These models consume tokens during the thinking process itself, and on complex tasks can generate thousands or even tens of thousands of reasoning tokens far beyond what the user actually sees — making them a new major cost driver.
• Always-on execution: Agent systems often run continuously via cron jobs, consuming tokens every minute like a faucet left open. A typical agent workflow also includes multiple rounds of model calls, re-ingesting tool call results, and reflection steps — each iteration stacking up the token count, pushing cost curves nearly exponential.
• Architectural mismatch: The core question with many agent systems is — why use a large model to do something traditional technology could handle just fine?

A classic example is data extraction. If the task is simply pulling data from a database, SQL and a relational database are obviously more efficient — that kind of workload can run on a Raspberry Pi costing a few dozen dollars, while calling a large model might require GPU infrastructure worth tens of thousands. Architects should seriously ask: which tasks belong in a relational database? Which steps actually require a large language model? And how should call frequency be reasonably designed?

The more pervasive waste comes from redundant work: within a single organization, multiple employees may each spin up identical agents performing identical tasks — essentially burning the same token costs over and over. Many organizations are simply "flushing money down the toilet, just because it's called artificial intelligence."

The Fundamental Question of AI Business Models

A sharp judgment is spreading through the industry: the core challenge facing companies like OpenAI and Anthropic isn't insufficient model capability, competitive pressure, or primarily the regulatory environment — it's that their business models may simply not pencil out.

The structural roots of the AI business model dilemma: The costs of training and running large models are primarily composed of compute (GPU/TPU rental or depreciation), electricity, and talent — all highly rigid expenditures that can't be quickly reduced when revenue falls. At the same time, the large model market is undergoing textbook commoditization: when Meta open-sourced the LLaMA series and DeepSeek replicated GPT-4-level performance at dramatically lower cost, the scarcity value of model capability dropped sharply, forcing commercial players to keep cutting prices to maintain market share. This creates a dangerous scissor effect: per-unit revenue is compressed by competition, while the infrastructure costs supporting the service are rising in absolute terms due to scale expansion.

The logic chain goes like this:

• AI should become more valuable as adoption deepens, but in practice it becomes harder to monetize as competition intensifies;
• Competition grows fiercer, while the fees that can be charged continue to decline;
• The hardware costs required to deliver outcomes either stay high or keep rising.

Revenue under constant pressure, costs stubbornly rigid, margins squeezed from both sides — that's why even seasoned industry veterans "can't figure out how this math works."

Historical context helps: from Windows 95 to 1999, the internet made its value felt by nearly everyone in just four years. Broadband adoption and the smartphone launch each brought visible life changes within the same timeframe.

Why is AI value perception slower than the internet? The internet's value was felt quickly and universally because the value chain was incredibly short: browser → webpage → information. Users needed virtually zero learning curve to feel the core benefit of "accessing information I couldn't access before." Smartphones were similar — the physicality of the device and the convenience of always having it on you made value perception almost immediate. Enterprise AI's value chain is far longer: purchase API access → build infrastructure → train/fine-tune → integrate existing systems → redesign workflows → wait for ROI to materialize. Every step can become a bottleneck. This explains why even as model capability genuinely improves, ordinary enterprises' perception of value remains frustratingly vague.

Nearly as much time has passed since ChatGPT sparked this wave, and the actual value that ordinary users and businesses can perceive still seems blurry around the edges.

A Measured Assessment

It's worth noting that some of the more pessimistic forecasts about AI companies carry an emotional charge, and readers should think independently.

But strip away the emotion, and several factual signals deserve serious attention: multiple tech executives are publicly saying token costs must fall dramatically; multiple leading tech companies are simultaneously pulling back AI budgets; and token waste in agentic architectures is a real, tangible pain point in engineering practice.

Seen from this angle, the current "budget reversal" may not be the end of AI — it may be the necessary inflection point where the industry transitions from "arms-race expansion" to "disciplined operations." The companies that survive and turn a profit may not be the ones burning the most tokens, but the ones who best understand how to allocate tasks appropriately between SQL, caching, and large models — making every token count.

Whether the AI bubble will burst remains an open question. But the cost equation is one the industry will eventually have to solve.

Key Takeaways