Turns, Not Observations: The True Cost Structure of Coding Agents and Cache Optimization

Coding agents operate in a repeated loop: modify code, run tools, inspect results, decide the next step. Researchers call the design of this process "loop engineering," covering context management, model selection, compression strategies, and termination conditions. Existing work on efficiency has largely focused on compressing the observations returned by tools, or routing some turns to cheaper models. But an analysis of 600 public trajectories offers a different answer: what actually determines cost is the number of turns, not the size of observations.

The Cost Structure Revealed by 600 Trajectories

The study covered two agent scaffolds and two language models, totaling 600 public coding-agent trajectories. The first counterintuitive finding: observations are far smaller than expected. The median observation is only 118 to 286 tokens. The large build logs often cited as evidence that "compression is mandatory" actually fall near the 99th percentile—they are extreme cases, not the daily norm.

The more critical finding comes from caching. The analysis shows that 98.6% of input tokens are served by a prefix cache. This means that estimating loop cost at flat input pricing overestimates the true cost by roughly 7x on average under near-ideal cache conditions. In other words, many concerns about "observations being too expensive" are built on a billing model that has already been substantially diluted by caching.

Prefix cache is a prompt caching mechanism in large-model inference services. In multi-turn dialogues or agent loops, each turn's request contains a large prefix identical to the previous turn (system prompt, conversation history, files already read, etc.). The inference server retains the key-value cache (KV cache) corresponding to this prefix, so the next turn only needs to compute the small number of newly added tokens; the cache-hit portion is billed at a rate far below standard input, typically an order of magnitude lower or even less. This is why the article emphasizes that "98.6% of input tokens are served by the prefix cache": the true marginal cost of a loop falls mainly on the incremental tokens that miss the cache, not on the accumulated context total. Understanding this is essential to grasping why estimating loop cost with cache-free flat input pricing is systematically distorted.

Turns Explain 84% of Cost Variance

The study performed a variance decomposition on cost drivers, with fairly clear results:

• Turn count explains 84% of the variance in logged trajectory cost
• Mean observation size explains only 4%

The most telling cases are the most expensive trajectories: they have the smallest observations but the most turns. This correlation points to a judgment: observation-focused compression methods target a secondary cost factor; the truly larger, yet almost systematically unstudied, opportunity lies in "unnecessary turns."

The authors remain restrained: they do not prove that turns can be removed while keeping task success rates unchanged. This is correlational evidence, not a causal conclusion.

Variance decomposition is a statistical method for measuring how much of the variation in an outcome each of several candidate variables can explain. Here, the total cost of each trajectory is the dependent variable, and turn count, mean observation size, etc., are independent variables, yielding shares like "turn count explains 84%, observation size explains 4%." The key point is that this gives correlation, not causation—high explanatory power only means the variable moves strongly with cost, not that intervening on it would reduce cost. This is precisely why the authors repeatedly stress they have not proven turns can be removed while preserving success rate: in coding agents, turns are often jointly determined by task complexity, codebase size, and failure retries; directly cutting turns may simply shift cost onto the success rate.

Model Delegation Has a Verifiable Break-Even Point

Building on the cost structure above, the study derives and empirically validates a closed-form break-even condition for model delegation. The conclusion: in half of the evaluation settings, delegation actually increased cost—and these increases appeared right near the predicted boundary. This provides counter-evidence to the intuition that "routing simple turns to a cheaper model always saves money."

The accompanying tool TurnCost allows re-scoring a trajectory corpus under alternative cache, pricing, compression, and routing strategies. It turns the question "if we changed strategies, how much would this batch of trajectories cost?" into a batch-computable problem.

Closed-form break-even condition refers to an analytical formula that directly gives the criterion for "under what conditions delegation is cheaper," without having to trial-compute each trajectory. Intuitively, handing simple turns to a cheaper model always seems worthwhile, but delegation itself brings overhead: the prompt and output of the delegation call, possible retries, and the probability that a less capable delegated model increases subsequent turns. When the unit-price savings from the cheap model fail to cover these added costs, delegation loses money. The article's statement that "in half of the evaluation settings delegation actually increased cost, concentrated near the predicted boundary" is precisely an empirical test of this threshold—results near the boundary are most likely to flip due to small changes in cache hit rates, task distribution, and other conditions.

Implications for Agent Engineering

The value of this work lies not in offering a definitive optimization plan, but in re-ordering the priorities of optimization targets:

1. Check caching first, then pricing: before discussing loop cost, confirm prefix cache hit rates—otherwise the pricing model can be off by multiples
2. Count turns first, then observations: shift attention from "observations are too large" to "why are so many turns needed"
3. Delegation is not default savings: routing to a cheaper model has a clear break-even boundary and must be calculated per scenario, not by intuition

Loop engineering is still a young field, but this analysis at least shows: when optimizing agent cost, the most conspicuous variable is often not the most important one.

One-line Summary

An analysis of 600 trajectories shows that agent loop cost is driven mainly by turn count, not observation size or model choice.

Paragraph Summary

An analysis covering two agent scaffolds, two language models, and 600 public trajectories re-orders the priorities of coding-agent cost optimization. The data show a median observation of only 118 to 286 tokens, with the often-cited giant build logs actually at the 99th percentile; 98.6% of input tokens are served by the prefix cache, and estimating loop cost at flat input pricing overestimates by roughly 7x on average. Variance decomposition shows turn count explains 84% of cost variance while mean observation size explains only 4%, and the most expensive trajectories have the smallest observations and the most turns. The study also derives and validates a closed-form break-even condition for model delegation, finding that in half of evaluation settings delegation increased cost. The authors stress this is correlational evidence and does not prove turns can be removed while preserving task success rate.