Why Are AI Coding Assistants So Expensive? The Real Bill Behind the Harness

Intelligence Is Getting Cheaper, So Why Are Bills Getting Higher?

Since OpenAI launched its first reasoning model O1, the price of AI intelligence has changed dramatically. Using GPT-5.6 as a benchmark, equivalent or even superior performance now costs roughly 2 to 3 times less than it did two years ago. This trend somewhat validates the "Jevons Paradox" — when a resource becomes cheaper and more accessible, overall consumption of it actually increases dramatically. This paradox was first proposed by British economist William Stanley Jevons in 1865, who observed that after James Watt improved steam engine efficiency, Britain's coal consumption actually increased significantly rather than decreased. In AI, this paradox is particularly pronounced: as inference costs dropped from tens of dollars per million tokens in 2023 to less than $1 in 2026, developers began embedding AI into more scenarios — expanding from single Q&A to continuously running Agent workflows, multi-turn code reviews, automated test generation, and more. Each individual call is indeed cheaper, but total call volume grows exponentially, and the final bill actually goes up. Intelligence has become readily accessible, and more and more people are using AI.

That said, open-source models are catching up at breakneck speed. In the past, open-source models typically lagged behind closed-source models by 6 to 18 months in performance, and didn't truly become competitive until 2025. Entering 2026, with the rise of models like Kimi K3, Qwen 3.8 Max, DeepSeek V4, and GLM 5.2, the open-source camp has secured prominent positions on major leaderboards, becoming viable alternatives to closed-source models. Open-source models have been able to close the gap rapidly thanks to several key technical breakthroughs: the maturation of Mixture of Experts (MoE) architectures enables the community to train ultra-large parameter models at lower computational costs; the proliferation of high-quality synthetic data pipelines means training data is no longer the exclusive moat of closed-source vendors; and the open sharing of inference optimization techniques (such as speculative decoding, KV cache compression, and quantized deployment) has significantly reduced performance loss in actual deployment of community models. The success of these models is essentially the result of the open-source collaborative ecosystem simultaneously reaching critical mass across data, algorithms, and engineering.

However, a key fact that many overlook is this: the cost of the model itself and the cost of the Harness (framework/shell) wrapped around it are two completely different things. This is the fundamental reason why many developers see their bills skyrocket without understanding where the money is going.

What Is a Harness? And Why Is It a "Hidden Money Shredder"

Most people use AI through chat interfaces, where the Harness is very "thin" — you're essentially talking directly to the model. This is still the most mainstream way to use AI.

But a small group of early adopters and power users work with much "thicker" Harnesses, such as Claude Code, Cursor, OpenCode, Codex, Cline, and other coding assistants. In these tools, you're no longer directly conversing with the model — instead, you work indirectly through an external intermediary: the Harness.

From a technical architecture perspective, a Harness is essentially an orchestration layer that sits between the user and the foundation model, responsible for managing context, tool calls, state tracking, and multi-turn interaction logic. A typical coding Harness contains the following components: a system prompt (defining the role and available tools), tool definitions (API schemas for file read/write, terminal execution, search, etc.), a context manager (deciding when to truncate, compress, or cache conversation history), and an execution sandbox (safely executing model-generated code). The key to understanding this architecture is that every component contributes additional token overhead, and this overhead is almost completely invisible in the user interface.

It's precisely this middle layer that causes costs to jump dramatically. According to Anthropic's data, simply by using an Agent or multi-Agent system, your spending could be 4 to 15 times what it would cost to call the model directly.

The author demonstrated this gap with a minimal experiment. He wrote a short prompt of only about 25 tokens: asking AI to create a file containing the alphabet. Even at Opus 5 pricing, this prompt plus the model's roughly 500-token response would cost just 1 cent for a direct API call.

But when he fed the same prompt into Claude Code, the total interaction cost immediately jumped to 37 cents — nearly 30 times more expensive.

System Prompts: The "Toll Booth" to Reach the Model

The biggest culprit behind this price explosion is the overhead of the Harness layer. Tools like Claude Code, OpenCode, Cline, and Cursor attach an additional 8,000 to 30,000 tokens to every message sent to the model. It's like a toolkit you must pay for before you can even reach the model.

This additional overhead is essentially the System Prompt. Every Harness has a system prompt, and Claude Code's is particularly long because it comes packed with features out of the box — every feature, every command needs to be described in detail within the system prompt so the model understands how to interact with the Harness.

It's worth noting that system prompt length isn't simply a case of "shorter is better." Longer system prompts typically mean richer tool descriptions, more precise behavioral constraints, and more comprehensive error handling guidance. For example, Claude Code's system prompt is around 20,000-30,000 tokens long because it needs to describe the precise calling formats, parameter constraints, and edge case handling for dozens of available tools (bash execution, file editing, search, Git operations, etc.). This is a classic engineering trade-off: the more detailed the system prompt, the higher the model's tool-calling accuracy and the lower the probability of error-induced retries, but the higher the base cost per call. Lightweight Harnesses like Pi opt for minimal system prompts, at the cost of potentially needing more retry rounds to complete complex tasks.

Interestingly, even when using the same Opus 5 model with the same prompt, different Harnesses produce wildly different costs:

• Claude Code: ~37 cents
• OpenCode: ~11 cents
• Cline: ~7 cents
• Pi: ~2.5 cents

The gap is so large that "choosing the right Harness" is itself a form of cost optimization. With open-source tools like Cline, you can even go directly to their GitHub repository to read the length and structure of their system prompt and understand how it works under the hood.

How Agent "Back-and-Forth Oscillation" Amplifies Costs

Many people have a misconception about Agentic applications, assuming the prompt is sent once, the model replies once, and it's done. That's true in chat applications, but Agent applications work completely differently.

In Agent mode, a "back-and-forth oscillation" occurs between the Harness and the model — like a recursive loop, going back and forth repeatedly until the model determines the task is truly complete. Using the "create alphabet file" example:

1. The request is sent to the model; the model decides to first check if the file already exists;
2. The Harness checks the computer and sends the result back to the model;
3. The model then tells the Harness to create the file;
4. The Harness creates the file and reports back;
5. The model finally tells the Harness to write the content.

Even for such a simple task, there are numerous round trips. For longer tasks like "build a complete website from scratch" that might run for 10 to 20 minutes straight, imagine how many round trips occur in between. The critical point is: every round trip carries the full system prompt as a "toll booth," multiplying costs accordingly.

From a technical implementation perspective, this round-trip pattern stems from a fundamental limitation of current LLMs — they are stateless. The model itself retains no contextual memory; every call must resend the complete conversation history (including the system prompt). Although some cutting-edge Harnesses have begun implementing context compression and selective history trimming, the system prompt — as the foundational instruction defining model behavior — can hardly be trimmed, making it an unavoidable fixed cost in every round trip.

Prompt Caching: Saves Money, Yet Also a Money Printer for Providers

Is there a way to reduce these costs? The answer is Prompt Caching.

When we send large payloads to the model, if we can reuse previously computed and stored caches, we save both time and money. To understand why Prompt Caching is effective, you need to understand the underlying mechanism: in the Transformer's autoregressive generation process, the model needs to perform attention computation over all preceding tokens for every new token generated. KV Cache (Key-Value Cache) stores the Key and Value vectors of previous tokens in GPU memory, avoiding redundant computation. For Prompt Caching, the principle is similar but operates on cross-request reuse: when multiple requests share the same prefix (such as a system prompt), the server can retain that prefix's KV cache in GPU memory, and subsequent requests start computation directly from the cached position, skipping forward propagation for the cached portion. This not only reduces computation (saving GPU power) but also lowers Time to First Token, which is particularly critical for the frequent, short-interval requests in Agent scenarios.

This is already very common among inference service providers and is especially important for Harnesses. Because servers temporarily retain KV caches, they can save substantial costs for users who want to maximize their subscription or API quotas.

This is why Anthropic's pricing page shows different prices for Prompt Caching. Anthropic even offers Prompt Caching for up to 1 hour (at a higher price), letting Harnesses save costs on long tasks like website building or deep research.

From an infrastructure perspective, Prompt Caching may be quite a profitable business for Anthropic. An NVIDIA H100 GPU rents for approximately $3 per hour. This GPU has 80GB of HBM3 memory with 3.35TB/s bandwidth. Depending on KV cache quantization method (FP16, FP8, or INT4), compression ratios can reach 2-4x, meaning theoretically one H100 could fit the entire 1-million context window's KV cache. This means that even if Anthropic splits this 1-million context window "workspace" into 12 portions, each cached for 5 minutes, a single H100's hourly revenue from Prompt Caching alone could reach approximately $70 — an impressive margin. Of course, the biggest assumption here is whether 1 million tokens of context can actually fit into a single 80GB H100.

As for 1-hour cache writes, since reserving GPU time for a full hour is expensive, the author speculates that Anthropic may evict idle memory from GPU to nearby SSDs (long-term storage is cheaper). NVMe SSDs offer approximately 7GB/s throughput and relatively low per-GB costs (~$0.1/month), making them ideal as a secondary cache tier for GPU memory. When the cache is needed again, it's reloaded to GPU on demand — which also explains how they can offer relatively low write pricing of about $10 per million input tokens, while the potential additional latency during reload is acceptable for non-real-time scenarios.

Subscriptions: Corralling Users Into the "Walled Garden"

While these mechanisms are fascinating, ordinary users don't actually want to be calculating costs while coding. The common approach vendors use to retain users is offering subscription plans at the model layer, letting Harnesses operate within the subscription quota.

ChatGPT, Claude, Gemini — all these frontier applications offer subscriptions. This way, you don't have to worry mid-workflow about how much the Agent is burning; as long as the quota refreshes periodically, you can focus on the AI use case itself.

For example, Anthropic's Pro plan might bundle all its models at $20/month, letting users freely use Claude Code, Claude Cowork, Claude Chat, and other Harnesses while switching between Opus, Sonnet, and Haiku. OpenAI does the same — a ChatGPT Plus subscription covers ChatGPT, Codex, and more, with multiple models sharing a single metering system.

The business logic of subscriptions is essentially an "insurance model": vendors bet that most subscribers' actual usage is far below that of power users, subsidizing power users' excess consumption with light users' "leftover quota," while maintaining user stickiness through lock-in effects (switching costs become extremely high once users are accustomed to a specific Harness's workflow). This is also why it's called a "walled garden" — once your workflow is deeply bound to a vendor's Harness ecosystem, the hidden costs of migration may far exceed the explicit subscription fees.

Understanding the Cost Structure Is the Only Way to Truly Save Money

The greatest value of this content is breaking down the vague anxiety of "why is AI coding so expensive" into three clear cost sources: heavy system prompts, the number of Agent round trips, and whether Prompt Caching is being utilized.

For developers, this brings several practical insights:

• For the same task and same model, switching to a lighter Harness can mean 5 to 15x cost difference;
• For long tasks, always verify whether your tool has Prompt Caching enabled;
• Subscriptions are convenient but also mean you're corralled into a specific vendor's "walled garden";
• For cost-sensitive teams, consider building custom Harnesses that trim unnecessary tool descriptions from the system prompt, keeping only the minimum feature set needed for the current task;
• Monitor the number of round trips and total token consumption for each Agent execution, establish cost baselines, and only then can you identify abnormal spending.

As open-source models continue climbing the leaderboards toward closed-source parity, flexible subscription options that aren't tied to a single vendor may become power users' new weapon against the "hidden money shredder." Understanding the mechanisms behind your bill is a lesson every heavy AI user must learn in this era.

(Note: This article is based on a single video source. Some cost estimates are the author's personal calculations, and actual figures may vary based on vendor strategies.)