Anthropic's Compute Economics Dilemma: Why Are Marginal Costs Rising Instead of Falling?

Anthropic's compute profit margin worsened when it should have improved, revealing structural tensions in frontier AI commercialization.
Anthropic faces a counterintuitive dilemma: at a stage when compute costs should decline with scale, its compute profit margin is deteriorating. The article analyzes three cost drivers: inference demand shifting to long context, multi-turn reasoning, and Agent tasks; training and inference competing for the same GPUs; and API price cuts outpacing cost reductions. This is not an isolated case but a microcosm of the frontier model track—compute economics depend on demand structure, resource scheduling, and pricing power rather than improving automatically with technology.
Anthropic is facing a counterintuitive predicament: at a stage when the industry broadly expects compute costs to decline with scale, its compute profit margin is deteriorating. This signal comes from tech media observations of the company's economic model, and it reflects an underestimated structural challenge facing leading AI labs in their commercialization efforts.

The Compute Ledger That Should Be Improving
According to the classic logic of the cloud computing and chip industries, unit token compute costs should gradually decline as model scale expands, hardware iterates, and inference optimization advances. Anthropic's Claude series has gone through multiple rounds of iteration over the past period, theoretically accumulating ample room for optimization.
But the reality is quite the opposite. According to the original report, just when Anthropic's compute profit margin should have been improving, it worsened instead. This means revenue growth failed to outpace the growth of compute expenses, or a mismatch emerged between pricing strategy and cost structure.
This kind of "reverse scale effect" is uncommon, and it points not to a simple operational efficiency problem but to a deeper tension in the business model.
Compute profit margin refers to the proportion of profit remaining after deducting compute (computing resource) costs as a share of revenue, and it is a core metric for measuring the unit economics of an AI lab. Its difference from gross margin lies in that compute costs include both the GPU hours and cloud service fees consumed during inference and the fixed investment amortized from training models. When revenue grows faster than compute expenses, the ratio rises; otherwise, it deteriorates. Understanding this metric requires distinguishing between "unit compute price" and "average compute consumption per token"—the former declines with hardware iteration, while the latter may rise due to changes in request structure, and the product of the two is the true cost curve.
Scale effect in the cloud computing industry usually refers to unit cost declines brought by expanding scale: stronger procurement bargaining power, higher utilization, and replacing general-purpose chips with specialized chips. But the scale effect of large model inference has special constraints: demand growth is often accompanied by increased request complexity, causing marginal costs to rise rather than fall. This is called the "reverse scale effect" or a "Jevons paradox"-style cost rebound.
Three Possible Cost Drivers
Changes in the Structure of Inference Demand
Large model use cases are migrating from simple question answering to long context, multi-turn reasoning, and Agent tasks. These requests consume far more compute than standard conversations and are more latency-sensitive. When the share of high-compute-density requests rises, the average marginal cost is naturally pushed up, even if the absolute price per unit of compute is declining.
Long context refers to a substantial increase in the input length a model must process in a single request, such as document analysis with hundreds of thousands of tokens. The computational load of the attention mechanism in the Transformer architecture has a quadratic relationship with context length, so the compute consumption of long-context requests grows far from linearly. Multi-turn reasoning involves mechanisms such as chain-of-thought and self-reflection that let the model perform multiple steps of internal computation before outputting, significantly increasing the number of output-side tokens. Agent tasks go further: the model needs to repeatedly call tools, observe results, and continue making decisions, so a single user request may trigger dozens of model calls. Together, these three types of scenarios raise the average compute consumption per request, and they are key to understanding the rise in average marginal costs.
Competition for Resources Between Training and Inference
Leading labs need to support both frontier model training and large-scale inference simultaneously. Compute investment on the training side is a rigid expense, while the inference side must ensure stability of user experience. When the two compete for the same batch of GPU resources, the cost allocation of the inference business is passively pushed upward.
The essence of GPU resource contention is scheduling conflict within the same cluster. Frontier model training typically requires thousands or even tens of thousands of GPUs running at full load for weeks, which is exclusive occupancy; inference services require low latency and high availability and need reserved elastic capacity to handle traffic peaks. When training tasks crowd out the cluster, inference can only rely on remaining fragmented resources or turn to more expensive on-demand cloud instances, causing the cost allocation on the inference side to rise passively. In addition, the hardware requirements of training and inference are not fully aligned—training favors high interconnect bandwidth, while inference values memory capacity and throughput efficiency more—and this heterogeneity makes resources difficult to fully reuse, further amplifying cost pressure.
Lag in Pricing Strategy
Price competition in the AI application market is intense, and API pricing is often cut before costs decline. If pricing is anchored to competitors rather than to one's own cost curve, a squeeze window forms in which "prices fall first and costs fall later."

Pricing power refers to a company's ability to adjust prices without losing customers. In the AI API market, leading vendors' pricing often references competitors rather than their own cost curves, creating an anchoring effect. When multiple vendors compete to cut prices and seize developer share, price cuts occur before cost optimization, compressing the profit window. Only vendors with differentiated capabilities (such as stronger reasoning quality, lower latency, or exclusive model capabilities) have room to maintain a premium. Vendors lacking pricing power are forced to follow price cuts before costs have fallen, and their compute profit margins therefore come under pressure. This is also why, when assessing the long-term sustainability of AI service providers, compute profit margin is more informative than absolute price.
What This Means for the Industry
Anthropic's situation is not an isolated case. It is more like a mirror for the entire frontier model track: compute economics do not improve automatically with technological progress but depend on the dynamic balance among demand structure, resource scheduling, and pricing power.
For developers, this means that a long-term downward trend in API prices is not a given; for enterprise buyers, a supplier's compute profit margin is an important indicator for assessing its service stability and long-term pricing capability, not a financial detail unrelated to them.
Whether Anthropic can reverse this trend will depend on its progress in inference optimization, its chip supply strategy, and whether it can establish sufficient pricing power in high-end scenarios. The original article does not answer these questions, but it puts the questions themselves on the table.
One-line Summary
Anthropic's compute profit margin worsened at a stage when it should have improved, revealing the structural tension in frontier AI commercialization.
Paragraph Summary
Anthropic is facing a counterintuitive predicament: at a stage when the industry broadly expects compute costs to decline with scale, its compute profit margin is deteriorating. The article analyzes three possible cost drivers: the structure of inference demand is shifting from simple question answering to long context, multi-turn reasoning, and Agent tasks, pushing up average marginal costs; training and inference compete for the same batch of GPU resources, causing inference cost allocation to rise; and API pricing has already been cut due to competition before costs decline, creating a profit squeeze window. This phenomenon is not an isolated case but a microcosm of the entire frontier model track—compute economics do not improve automatically with technological progress but depend on the dynamic balance among demand structure, resource scheduling, and pricing power. For developers and enterprise buyers, this means that a long-term decline in API prices is not a given, and a supplier's compute profit margin is an important indicator for assessing its service stability.
Related articles

How Did North Korea Become a Key Foreign Backer of Russia's War Machine? A Deep Dive from the Lex Fridman Podcast
Lex Fridman Podcast military analysis reveals the foreign support structure of the Russia-Ukraine war: North Korea once supplied 40% of Russian artillery ammunition and sent troops, Russia's war spending is 9%-10% of GDP but its economic structure is deteriorating, while Ukraine relies heavily on Western support and faces U.S. policy uncertainty.

Ressa's New Book: Why Tech Oligarchs Are a Threat on Par With Dictators
Maria Ressa explains in a Vox 'The Gray Area' interview why she calls Zuckerberg and Duterte both dictators, Facebook's role in the Philippines drug war, the tech oligarch accountability gap, and how ordinary people can fight back by auditing feeds, building open-source tech, and taking offline action.

A Black Hole Hiding at the Edge of the Solar System? An Alternative Explanation for Planet Nine
Could Planet Nine at the solar system's edge be a primordial black hole? Explore the origins, evidence, and controversy behind this orbital anomaly hypothesis.