Claude Haiku 5.5 Long-Context Billing Trap: A Node Guard Against 5x Bills

Why a 99K and a 101K Prompt Differ by 5x on Your Bill

In your code, a 99,000-token prompt and a 101,000-token prompt look identical. But on Claude Haiku 5.5, the latter costs roughly five times as much.

According to Anthropic's official pricing page, Haiku 5.5 charges $0.10 per million input tokens and $0.50 per million output tokens for prompts up to 100,000 tokens; once you cross that line, input jumps to $0.50 and output to $2.50. Note that output is also billed at the higher tier.

Let's do the math: for a request that outputs 1,000 tokens, a 99K prompt costs $0.0104, while a 101K prompt costs $0.0530. The prompt grew by only 2%, yet cost becomes 5.1x. At 100,000 requests per day (ignoring caching and batch discounts), the daily bill jumps from about $1,040 to $5,300.

Two Easily Overlooked Billing Details

Anthropic's long-context pricing docs hide two key points:

• "The prompt length of a request counts all input tokens, including cache reads and cache writes."
• "Requests that exceed the threshold are billed at the higher tier even if part of the prompt hits the cache."

This means a 90,000-token cached system prefix plus 15,000 tokens of new content counts as a 105,000-token prompt, and the cache-read rate itself rises from $0.01 per million to $0.05 per million. Prompt caching cannot help you bypass this line—only a shorter prompt can.

The threshold is calculated per request, and "up to 100,000" means exactly 100,000 tokens still counts as the base tier. The guard logic treats anything over 100_000 as the long tier.

Prompt caching is a mechanism in the Anthropic API: for long prefixes that repeat (such as system prompts and document libraries), the server can cache the computed results, and subsequent requests that hit the cache are billed at a lower "cache read" rate instead of being fully recomputed. Normally this saves significant money, but cache pricing has its own rate card: cache reads in the base tier are about $0.01 per million tokens, while in the long tier they rise to $0.05 per million, and cache-write rates are likewise tiered and usually higher than regular input. The key point is that tiering depends on the prompt's total token count, not the "uncached increment," so a large cached prefix plus a small amount of new content is still judged as the long tier overall. This also explains why this article stresses that caching can only lower unit prices, not change tier assignment—the only effective way to avoid it remains shortening the prompt itself.

Count Tokens Before Sending: The Core Design of the TypeScript Guard

Anthropic's TypeScript SDK exposes the token-counting endpoint as client.messages.countTokens. It accepts the same model, system, messages, and tools as the Messages API and returns { input_tokens }. Three facts determine how the guard is written:

• Free but separately rate-limited: 5,000 requests per minute on the Start tier, not counted against the message-creation quota.
• The count is an estimate: the official docs acknowledge that "the actual number of input tokens used when creating a message may differ slightly." So the guard should target 97,000 rather than 100,000, leaving headroom.
• Haiku 5.5 uses a new tokenizer: the same text yields about 30% more tokens than on Haiku 4.5. Counts you measured on the old model are invalid and must be re-measured.

The guard centralizes request construction in a single buildParams function, ensuring that "what you count" is "what you send":

// src/guard.ts
export const MODEL = "claude-haiku-5-5";
export const TIER_LINE = 100_000; // over this = long tier
export const BUDGET = 97_000; // headroom: counts are estimates

export type Chunk = { id: string; score: number; text: string };

export type Params = {
  model: string;
  system: string;
  messages: Anthropic.MessageParam[];
};

export type Counter = (p: Params) => Promise<number>;

Designing Counter as a plain function type is deliberate. In production you pass in apiCounter(client); in unit tests you pass a fake implementation, so the pruning logic runs without a network or API key. Params deliberately omits max_tokens, which belongs to the create call, not the count call.

Tokenizer: the component that splits raw text into tokens; different models may use different versions. A token is not a fixed number of characters: English averages about 3-4 characters per token, while Chinese, code, and special symbols tend to be more token-hungry. When a vendor upgrades the tokenizer, the token count for the same text changes—this is exactly the source of "Haiku 5.5 uses about 30% more than 4.5." Token count directly determines the billing tier and context-window usage, so any count obtained on an old model cannot be reused across models. In engineering practice, converting text length to tokens can only approximate a ratio (e.g., 1M tokens ≈ 2.5M Unicode characters), and this is an average, not an upper bound—suitable only for skipping requests clearly below the threshold. To accurately determine whether a request crosses tiers, you must call the official countTokens endpoint to measure it.

Rank-Based Pruning: Binary Search for the Largest Prefix That Fits

The chunks returned by the retriever are already sorted by relevance. When all chunks exceed the budget, the guard keeps the longest prefix of the highest-ranked chunks that fits. Since token count grows monotonically with the number of chunks, binary search needs only about log2(n) counts instead of one count per dropped chunk.

// ranked: best chunk first
export async function fitToBudget(
  count: Counter,
  q: string,
  ranked: Chunk[],
  budget = BUDGET,
): Promise<Fit> {
  const all = await count(buildParams(q, ranked));
  if (all <= budget) {
    return { chunks: ranked, tokens: all, dropped: 0 };
  }
  // invariant: lo chunks fit, hi chunks do not
  let lo = 0;
  let loTokens = await count(buildParams(q, []));
  if (loTokens > budget) {
    throw new OverBudget(`${loTokens} tokens with no docs`);
  }
  let hi = ranked.length;
  while (hi - lo > 1) {
    const mid = Math.floor((lo + hi) / 2);
    const t = await count(buildParams(q, ranked.slice(0, mid)));
    if (t <= budget) {
      lo = mid;
      loTokens = t;
    } else {
      hi = mid;
    }
  }
  return { chunks: ranked.slice(0, lo), tokens: loTokens, dropped: ranked.length - lo };
}

On the happy path, each request costs just one extra count. When over budget with 40 chunks, it's two counts plus about six search counts. Each count is a round trip and adds latency. If latency-sensitive, you can skip counting for obviously tiny prompts—according to the official model overview, under this tokenizer 1M tokens is roughly 2.5M Unicode characters. But that ratio is an average, not an upper bound, so it can only be used to skip prompts far below the threshold.

There are two edge cases the caller must handle. OverBudget means the system prompt plus the question alone exceeds the limit—an upstream bug that pruning chunks cannot fix. If the top-ranked chunk is itself enormous, lo may drop to zero and the context becomes empty; check fit.chunks.length before deciding whether to answer.

Binary search is suited to locating a boundary on a monotonic sequence: if the first k chunks fit within budget, then the first k-1 necessarily also fit, and token count is monotonically non-decreasing in the number of chunks, so "the maximum prefix length that fits" forms a clean binary-search boundary. The algorithm maintains two boundaries—lo (known to fit) and hi (known not to fit)—counts the midpoint and shrinks the interval each time until hi − lo = 1, at which point lo is the answer, with O(log n) complexity. Compared to the linear approach of dropping chunks from the end and recounting one by one, binary search needs only about 6 counts for 40 chunks, bringing the latency of each count's network round trip down to an acceptable level. The precondition is that monotonicity holds—i.e., appending a chunk never reduces the token count, which is naturally satisfied when the prompt is simply concatenated.

Log the Tier the API Actually Bills, Not Just What You Counted

The count is an estimate; the usage in the response is what you actually pay for, so record both. Here's the trap: input_tokens includes only the tokens "after the last cache breakpoint," and the total is cache_read_input_tokens + cache_creation_input_tokens + input_tokens. A dashboard that looks only at input_tokens might show 15,000 for a request billed at the long tier, leading you to misjudge.

The billedPrompt function sums the three, and tier determination uses it rather than input_tokens. The ask function runs the guard, sends the pruned request, and prices it from the returned data, logging counted (local count), billed_prompt (actual billed prompt), tier (tier), dropped (number of dropped chunks), and est_usd (estimated USD).

est_usd prices all prompt tokens at the uncached rate, so it overestimates on cache reads and slightly underestimates on cache writes (cache-write rates exceed base input). Its purpose is to catch tier jumps on a chart; bill reconciliation is left to the billing system. Alerts should watch two things: any log line with tier: "long", and a steadily rising dropped count—the latter indicates that retrieved content exceeds what the budget can hold.

The usage field and cache breakpoints need to be clearly distinguished: the usage in the response returns input_tokens, cache_creation_input_tokens, and cache_read_input_tokens separately. When a request carries a cache_control marker, the prompt prefix before it is cached, and only the tokens after it count toward input_tokens. So looking only at input_tokens reflects just the new portion after the breakpoint; the full prompt length must sum all three. This also means a cache hit does not reduce the prompt's "total length," only the unit price of each segment. When doing cost monitoring and tier determination, you must use the sum of all three (this article's billedPrompt) rather than input_tokens; otherwise the dashboard will systematically underestimate real usage, especially for requests that just cross the 100K line, which are easily misjudged as the low tier.

Why Pruning Beats Switching Models

The obvious alternative is to send long prompts to another model. But on price alone, that doesn't solve the problem. Haiku 5.5's long-tier $0.50/$2.50 is still lower than Haiku 4.5's $1/$5 and Sonnet 5.5's $2/$10. Even accounting for Haiku 4.5's older tokenizer making the same text about 23% shorter—so a 101,000-token Haiku 5.5 prompt is about 78,000 tokens there, at an input cost of about $0.078—that's still higher than $0.0505. Switch models when you need a stronger model; when it's just a length problem, keep using the cheaper one.

When pruning would hurt answer quality (say, a summarization task that must read the whole document), there are three paths: split the input into below-threshold blocks, summarize each, then merge (more requests, but each at the base tier); use the Batch API, where the long-tier price is halved to $0.25/$1.25 (if the task can wait); or deliberately accept the long tier and have your logs record it truthfully.

Put the Budget Check into CI with Vitest

Runtime pruning catches drift per request; a CI check catches it in the very PR that introduced the problem. Keep a fixture of the "worst prompt your config can produce": the longest question you accept, plus topK chunks at maximum size, drawn from a real corpus. Then count and assert it doesn't exceed BUDGET.

In tests, use test.runIf(hasKey) so network tests are skipped in local environments without a key. Also add a guard test: under CI, assert that ANTHROPIC_API_KEY exists, because GitHub Actions always sets CI=true, so a pipeline missing the secret fails outright instead of silently skipping the check. The count call is free, and running it once per PR only consumes rate-limit headroom.

The pruning logic itself is unit-tested with a fake counter at roughly four characters per token, without touching the network. When the budget test goes red, the PR that raised topK or chunk size must explain why the extra context is worth five times the price. Sometimes it genuinely is—then a reviewer signs off on the cost before release.

One-line Summary

Claude Haiku 5.5 bills at 5x once you exceed 100K tokens, so you must count and prune before sending and log the actual billing tier.

Paragraph Summary

The article exposes the tiered-billing trap of Claude Haiku 5.5: once a prompt exceeds 100,000 tokens, both input and output unit prices jump 5x, and cache hits cannot bypass the threshold because tiering is based on the request's total token count. The author presents a TypeScript guard: use countTokens before sending, targeting 97K to leave estimation headroom; exploit the monotonic growth of tokens with chunk count to keep the highest-ranked largest prefix that fits via binary search, recording the dropped count when over budget. At the logging level, it stresses that the actual billed prompt is the sum of cache_read, cache_creation, and input_tokens—looking only at input_tokens severely underestimates and misjudges the tier. Finally, it argues pruning is more cost-effective than switching models, and uses a Vitest fixture to catch cost drift in CI.