Real-World Test: Claude Haiku 5.5 Costs 12x More Than GPT-6 Luna

An Unexpected Backfire on a "Cheap" Model

When Anthropic launched Claude Haiku 5.5 as their most affordable model, many developers hoped it would finally shake the Claude family's long-standing reputation as a "token gobbler." But a Reddit user put that claim to the test with the classic voxel pagoda benchmark — and the result was startling: for the exact same task, Claude Haiku 5.5 cost 12 times more than its direct competitor, GPT-6 Luna.

This finding is worth unpacking carefully, because it exposes a truth that's easy to overlook: a model being "cheap per token" and being "cheap to actually use" are two entirely different things. The numbers on a pricing page tell you almost nothing about what you'll actually spend in a real workload.

The Test Data: Where Does the Gap Come From?

The tester chose the voxel pagoda generation task — a well-known benchmark in the developer community. Both models were accessed through the atomic.chat platform via their respective subscriptions, and both were configured at xhigh reasoning intensity. Here's what the numbers looked like:

Model	Reasoning Intensity	Input Tokens (Cached)	Output Tokens	API Equivalent Cost
Claude Haiku 5.5	xhigh	268.8M (262.7M)	4.46M	$24.96
GPT-6 Luna	xhigh	64.5M (61.1M)	1.18M	$1.96

Two compounding gaps are visible in the data. First, raw token consumption: Haiku 5.5 used 268.8 million input tokens — over four times Luna's 64.5 million — with output tokens nearly four times higher as well. This confirms the tester's assessment: Haiku is still very much a "token goblin," and hasn't become the lean, efficient model many hoped for.

The Real Cost Trap: 5x Pricing After the 100k Context Threshold

If the gap were purely about consuming four times more tokens, you'd expect roughly a 4x cost difference — not 12x. The tester identified the true culprit: Claude Haiku 5.5's API price jumps 5x once the context exceeds 100k tokens.

In practice, this threshold is nearly impossible to stay under in agent workflows. In a continuous agent session, almost every round of conversation will accumulate well past 100,000 tokens — which means the vast majority of requests are being billed at five times the base rate. Multiply four times the token volume by the frequently triggered 5x pricing tier, and you arrive at that staggering 12x total cost difference.

GPT-6 Luna, by contrast, sets its premium-tier threshold at 272k tokens — nearly three times higher. More importantly, Luna's default configuration includes auto-compact, which automatically compresses context before reaching that threshold, proactively preventing the high-cost pricing zone from being triggered.

---

What is tiered context pricing? Tiered context pricing is a billing strategy now widely adopted by large model API providers. Vendors set a token-length threshold: requests below it are billed at the standard rate, while requests exceeding it switch to a higher rate. The business logic is straightforward — processing very long contexts demands disproportionately more compute (especially KV cache VRAM), so providers use price signals to encourage developers to keep context lengths in check. For single-turn Q&A applications, the threshold is rarely reached. But in agent scenarios, every tool call result, conversation turn, and system prompt gets appended to the context window and accumulates linearly across rounds — meaning nearly every request ends up in the premium pricing zone. The real-world impact of tiered pricing depends entirely on your application's context growth pattern, not the model's advertised price.

---

Practical Lessons for Developers

This comparison surfaces several insights worth keeping in mind when building real AI applications.

Pricing thresholds matter more than per-token rates. Many models use tiered billing — attractive rates at low context lengths, but non-linear cost spikes once you cross a certain token threshold. Before choosing a model, always find out where that threshold sits and whether your typical workload will regularly exceed it.

Agent workflows amplify every cost. A single Q&A interaction might never touch the 100k threshold. But an agent session continuously accumulates context. If your product involves multi-turn autonomous execution, your context growth pattern will largely determine your final bill.

Auto-compaction is a genuinely money-saving design choice. Luna's auto-compact isn't a minor convenience feature — it's a critical mechanism that actively applies the brakes before entering the high-cost pricing tier. Models without a similar mechanism require developers to implement their own context management to keep costs under control.

It's worth noting that this analysis is based on a single Reddit user's voxel pagoda test, and results may vary across different task types. But it clearly delivers one key takeaway: when evaluating model costs, don't just look at the price per million tokens. You need to account for token consumption efficiency, billing thresholds, and your actual workload patterns together. For any model that markets itself as "the cheapest," that full calculation is absolutely worth running yourself.

---

What is context compression (auto-compact)? Context compression refers to automatically summarizing or truncating conversation history or intermediate reasoning results when the context window approaches a certain threshold, reducing the token count for subsequent requests. The core trade-off is accepting a small loss of information in exchange for a significant reduction in billing costs. There are two main approaches: automatic compression built into the API layer by the model provider (such as Luna's auto-compact), and explicit compression managed by developers at the application layer — for example, periodically replacing raw conversation history with a summarized version. Provider-level compression is transparent to developers but offers less flexibility; developer-managed compression requires additional engineering effort but allows fine-grained control over when and how compression happens. For models without built-in compression, context will grow unboundedly unless developers actively intervene — continuously triggering the premium pricing tier.