Stop One Customer from Blowing Up Your OpenAI Bill: Per-Customer Cost Control

When you run an LLM API-based SaaS product, your vendor bill is usually just a single number. OpenAI and Anthropic let you set limits per organization or project, but those limits protect your account security, not your profit margin—they can't tell you which customer contributed to that spend, and they can't throttle a single person without affecting everyone else.

This loss of control shows up in two typical ways:

1. The runaway script user: someone scripts calls against your feature, or a client bug gets stuck in a retry loop, and a single account burns real money overnight.
2. The silent loss-making account: a customer on a $20 plan sends extremely long contexts every day, and their token cost far exceeds what they pay you. You can't see it because costs aren't attributed per customer.

Below is a pattern that solves both problems at once, plus the edge cases to watch out for.

1. Convert every call to dollars per customer

Every chat completion returns token usage. Convert it to dollars immediately, because token counts aren't comparable across models:

cost = (uncachedInput * inputPrice + cachedInput * cachedInputPrice + output * outputPrice) / 1_000_000

A few details that are easy to trip over:

• Cached input tokens: OpenAI reports them in usage.prompt_tokens_details.cached_tokens, and they're billed at a lower rate. Ignoring this will overestimate your costs.
• Dated model snapshots: model names like gpt-5.4-mini-2026-03-17 should resolve to the base model's price.
• Price table: keep it in one place, with source links and the date you read that price. Prices change. When the author checked OpenAI's pricing page on 2026-10-09, gpt-5.5 was listed at $5 per million tokens for input and $30 for output, and gpt-5.4-mini at $0.75 / $4.50. Verify before you rely on it.

Cached input tokens (prompt caching) is a discount mechanism offered by vendors like OpenAI for large repeated prefixes: when multiple requests carry the same prefix content (such as a system prompt, a long document, or few-shot examples), the vendor caches the computation of that prefix, and subsequent input tokens that hit the cache are billed at a lower rate, usually around one-tenth of the standard input price. For a SaaS product, this is both a money-saving opportunity and an accounting trap—if you count all prompt tokens at the standard input price, you'll significantly overestimate actual spend, causing downgrades or blocks to trigger too early. Conversely, if you apply the cached rate to non-cached tokens, you'll underestimate costs and let loss-making accounts slip through quietly. So when parsing usage, you must distinguish cached_tokens from regular prompt_tokens and multiply each by the corresponding rate. The cache isn't permanent—vendors usually have a TTL ranging from a few minutes to an hour, and the hit rate fluctuates with user behavior—so cost accounting should be based on the usage fields the vendor actually returns, not a hit rate you assume yourself.

2. Check before the call, record after

Before the call:  spent = store.get(customer, month)
         if spent >= budget        → block
         if spent >= 0.8 * budget  → switch to a cheaper model
Call the vendor
After the call:   store.add(customer, month, cost)

This check must live in the request path, before the call. A nightly cron job that emails you about losses is forensics after the fact, not protection.

In-request-path check means embedding the budget decision into the synchronous code flow that handles user requests, in the same execution chain as the actual vendor call, so that every call is based on the latest accumulated value. By contrast, a cron job only reads the bill at fixed points in time, and all spending between two runs falls into a monitoring blind spot—especially unable to handle sudden script abuse or retry storms. When implementing this, understand that the read-decide-call three steps are not atomic: under high concurrency, multiple requests can read the same not-yet-over-budget balance almost simultaneously and then all get let through. A single-process Node.js environment can mitigate this because the event loop executes serially, but in multi-instance or serverless architectures you must rely on external atomic operations. So this pattern is "soft protection," meant to prevent most runaway scenarios, not a financial-grade hard cap; if your business requires a strict ceiling, you need to reserve an estimated amount before the call.

3. Handle streaming responses correctly

When using stream: true, OpenAI only reports usage if you set stream_options: { include_usage: true }, and it appears in the final chunk. Wrap the async iterator, pass chunks through unchanged, and record the cost in a finally block so it still runs even if the client stops reading early.

Streaming responses (stream: true) return text chunk by chunk as Server-Sent Events, and token usage is not given in a single complete response like non-streaming. OpenAI requires explicitly passing stream_options: { include_usage: true }, after which the final chunk carries the complete usage object with an empty choices array. If you forget this parameter, the usage field simply won't appear, all cost records will be lost, and per-customer accounting will fail outright. Another pitfall is that the client may disconnect mid-stream or actively stop reading; if your recording logic is placed after the consumption loop, the code may be interrupted and skip accounting. The correct approach is to wrap the for await iterator with try/finally and put store.add in the finally block, ensuring it gets recorded whether the stream ends normally or exits early. Also be careful not to modify the content or order of chunks when passing them through, or you'll break the upper layer's SSE parsing.

4. Downgrade first, block second

Cutting off paying customers outright is a terrible experience. A gentler first step works better: after exceeding 80% of budget, route the same task to a cheaper model. Many users won't notice, and your margin is protected. At 100%, block with a specific error so your UI can display "You've reached this month's AI usage limit. Want to upgrade?" instead of throwing a vague failure.

5. Use storage that can handle concurrency

An in-memory counter is fine for a single process. But when facing multiple instances or serverless functions, use atomic increments, such as Redis's INCRBYFLOAT llmbrake:{month}:{customer} {cost}. Key by month so budgets reset without a cron job; add EXPIRE and old months clean themselves up.

Two honest caveats:

• Overshoot: concurrent in-flight requests may each pass the check and then overspend together, usually by the cost of those few calls. If you need a strict ceiling, you can reserve an estimated cost before the call and settle after.
• Storage failure: decide whether to fail open (keep serving, accept some risk of unmetered spend) or fail closed (protect spend but break functionality). For most products, fail open plus alerting is the right choice.

Fail open vs. fail close are the two trade-off strategies for rate limiting and quota systems when their own dependencies fail. When an external counter like Redis is unreachable, times out, or errors out, fail open means skipping the budget check and continuing to let requests through, at the cost of that period's spend going unmetered and carrying abuse risk, but user functionality isn't interrupted; fail close is the opposite, rejecting requests outright to protect costs, at the cost of normal paying users also experiencing service unavailability. The choice depends on the nature of the business: consumer-facing products that prioritize retention and experience usually choose fail open, paired with alerting and degraded logging, backfilling records or investigating manually after recovery; scenarios that are internal-facing, cost-sensitive, or at risk of abuse may choose fail close. Whichever you choose, failure events should be reported to your monitoring system to avoid silent failures. In practice, you can set up a short-lived local degraded cache as a compromise, or flag requests let through during the failure window for easier post-hoc attribution.

6. Compare cost against revenue

Once spend is attributed per customer, you can correlate it with what the customer pays you (e.g., a Stripe subscription). A table with customer, AI cost, revenue, and margin is usually enough to reveal which plans need repricing or lower caps.

Library version

The author packaged this pattern into LLMBrake, a small MIT-licensed TypeScript library (about 500 lines, zero runtime dependencies):

npm install llmbrake

import { createGuard, isBudgetExceededError } from "llmbrake";

const guard = createGuard({
  budgetUsd: (ctx) => (ctx.plan === "pro" ? 20 : 2),
  downgradeAt: 0.8,
  downgradeMap: { "gpt-5.5": "gpt-5.4-mini" },
});

const chat = openai.chat.completions;
const create = guard.wrap(chat.create.bind(chat));

const res = await create(
  { model: "gpt-5.5", messages: [{ role: "user", content: "Hello" }] },
  { customerId: user.id, plan: user.plan, feature: "chat" },
);
// When the limit is reached → throws BudgetExceededError

It covers cached token pricing, streaming usage, downgrade steps, typed block errors, metadata-only usage events, and a pluggable store that defaults to fail open. You can use it directly or copy the pattern. There's also an in-browser demo (real library, mock vendor): https://llmbrake.com/demo?ref=devto

Disclosure: the author built LLMBrake. A hosted dashboard (shared counters, alerts, per-customer AI cost vs. revenue comparison) is in early access at app.llmbrake.com.

Key takeaways

• Check the budget in the request path, before the call, rather than relying on a cron job afterward.
• Convert token usage to dollars immediately, and correctly distinguish cached input and dated model snapshots.
• Downgrade to a cheaper model before blocking, protecting both the paying customer experience and your margin.
• In multi-instance environments, use atomic counters like Redis to accumulate cost per customer, per month.
• Correlate per-customer attributed AI cost with revenue to identify plans that need repricing or caps.