The Real Cost of AI Agents: Input Tokens Are the Hidden Driver of Your Bill

A Widely Misunderstood Cost Assumption

When people think about the cost of running an AI Agent, the instinct is to focus on output tokens — after all, the content the model generates is what we actually want. But a set of real-world data from the DeepSWE benchmark has completely upended that intuition.

In a comparative experiment running DeepSWE on both Astra and DeepSeek V4.1-Flash, researchers found that input tokens outnumbered output tokens by a ratio of 174 to 1. In other words, for every single output token an Agent generates, it consumes 174 input tokens behind the scenes. This lopsided ratio is the key to understanding the true cost structure of AI Agents.

Why Are There So Many Input Tokens?

For coding-focused Agents like DeepSWE — systems designed for software engineering tasks — this ratio actually makes perfect sense. To complete a task, an Agent must repeatedly read the codebase context, conversation history, tool call results, error logs, and a large volume of other information. With each reasoning step, the model must "re-read" an enormous context window, while the actual decisions or code it produces are often just a handful of lines.

This "read a lot, write a little" pattern means that the cost structure of an Agent is fundamentally different from a traditional chatbot. If you optimize based on the intuition that output tokens dominate costs, you're likely directing your effort at the wrong target.

This phenomenon stems from the full-context pass-through approach adopted by most mainstream Agent frameworks. Unlike human conversation, where selective memory is possible, large language models are stateless — every inference call must include the complete history in the request. In multi-step software engineering tasks, context accumulates with every tool call (reading files, executing commands, capturing error output): what starts as a few thousand tokens in round one may balloon to hundreds of thousands by round ten, and every new inference must repackage everything that came before and send it back to the model. This mechanism is known as "snowball context," and it's the core reason Agents have a fundamentally different cost structure than single-turn conversations. Some frameworks attempt to truncate history using summarization or sliding windows, but this often comes at the cost of task coherence. Striking the right balance between cost and context completeness remains a significant challenge in Agent engineering.

Cache Hits: The Hidden Star of Your Bill

Even more noteworthy is the role of caching. The test data showed that 99.6% of all input tokens were cache hits. This means the vast majority of the context an Agent processes is the same information being read repeatedly across multiple interaction turns.

The critical insight: although cache hits are priced lower than regular input tokens, their sheer volume means they account for 60% of the total bill. That figure is striking — it means the central battleground for optimizing Agent costs isn't output tokens, or even "fresh" input tokens, but how efficiently you handle the cached content being read over and over again.

For developers and platform providers, this points to a clear optimization target: whoever can push the pricing and efficiency of caching mechanisms to their limits will hold the competitive advantage in the Agent cost race.

Prompt Caching is a feature now widely offered by mainstream LLM APIs. The principle: when the input prefix received by the model exactly matches a previous request, the cloud provider can reuse the previously computed KV cache (Key-Value Cache), skipping redundant attention calculations and charging a lower rate as a result. For example, Anthropic Claude prices cache-hit tokens at roughly one-tenth the cost of regular input tokens; OpenAI typically charges cache-hit tokens at a 50% discount. In Agent scenarios, system prompts, tool definition lists, and long code files remain unchanged across every round — making them natural candidates for cache hits. This explains why the cache hit rate reached 99.6% in the DeepSWE tests: most of the Agent's context is "old content," with only the latest tool results and model responses being truly new. Understanding this mechanism helps developers proactively structure their prompts — placing stable content first — to maximize cache hit rates.

Same Quality, 15x Cost Difference

The most striking finding from the experiment is the cost comparison. With identical task completion quality:

• Astra solution: $0.43 / task
• DeepSeek V4.1-Flash solution: $6.52 / task

A difference of more than 15x. This gap doesn't come from differences in model capability — it comes from cost structure, specifically how each solution prices and handles the high proportion of cached input tokens.

This also makes clear that evaluating the economics of an Agent solution can't stop at the model's headline price (e.g., cost per million tokens). You need to factor in the real-world workload characteristics: the input-to-output ratio, cache hit rate, and how cached tokens are actually billed. Two solutions that get the job done equally well can produce bills that differ by an order of magnitude.

Astra is a multimodal Agent framework and platform from Google DeepMind, designed for long-horizon, multi-turn tool-calling scenarios, with targeted optimizations for caching and streaming inference. DeepSeek V4.1-Flash is a lightweight, high-speed inference model from DeepSeek, known for low latency and relatively low standard token pricing. The difference in their listed prices doesn't necessarily translate directly to a difference in actual bills — and that's exactly the core argument of this article: when cached tokens make up the overwhelming majority of input, the discount rate and billing granularity of caching become the decisive cost variables, not the "price per million tokens" listed on the model's official page. This should prompt developers to run end-to-end cost benchmarks using workloads that reflect real usage, rather than relying on static price sheets for side-by-side comparisons.

What This Means for Developers

These findings come from a single benchmark, but the patterns they reveal are relevant to any team building Agent applications:

1. Audit your cost structure, not just your total spend. Understand what portion of your Agent bill comes from input, output, and cache respectively — that's how you find the real optimization levers.
2. Caching strategy is a core competency. When 99.6% of your input is cache hits, even small differences in cache pricing get amplified by massive volume into enormous gaps on your bill.
3. Don't be misled by the "output token" intuition. For long-context, multi-turn, tool-calling Agent workflows, the input side is almost always where the cost weight lies.

It's worth noting that these conclusions come from one specific benchmark (DeepSWE), and the exact numbers will vary with task type, context length, and implementation. But they offer a clear reminder: in the economics of AI Agents, intuition often misleads — data doesn't.