LLM API Cost Guide: How Developers Can Control Their AI Bills

Training a frontier large model costs tens of millions or even hundreds of millions of dollars — which is simply unrealistic for most teams, and unnecessary. The bill that actually needs to be calculated is: how much does it really cost to call a model through an API, given your own traffic volume?

This article provides a ready-to-use estimation method, a reference pricing table, and a Python script you can run on your own data.

The Basic Logic of LLM API Billing

Nearly all major providers charge by token, and quote prices in units of "per million tokens." Understanding the following dimensions is a prerequisite for controlling costs:

• Input tokens: system prompts, user messages, retrieved context, conversation history.
• Output tokens: content generated by the model, typically priced at 3–5× the input rate.
• Context length: some models charge a higher rate for extra-long context requests.
• Prompt Caching: reused prefixes (system prompts, product catalogs, style guides) enjoy substantial discounts.
• Batch API: non-urgent tasks typically get about half off.

The core formula can be written as:

monthly_cost = requests_per_month × (
    avg_input_tokens  / 1_000_000 × input_price +
    avg_output_tokens / 1_000_000 × output_price
)

Putting this formula into concrete numbers lets you quickly judge whether a given feature fits within budget.

What is a token: A token is the smallest unit of text the model processes, and is not strictly equal to "a character" or "a word." In English, roughly 1 token ≈ 4 characters (about 0.75 words). In Chinese, due to different tokenization, a single Chinese character usually occupies more than 1 token. This means the billing basis for the same content differs between Chinese and English, so when budgeting it's best to measure with the provider's tokenizer rather than making a rough estimate by character count.

Why charge by token instead of by call count: A model's compute cost scales approximately quadratically with sequence length (due to the attention mechanism) — the longer the input, the more expensive the inference. Token-based billing lets price reflect real compute consumption, and it also explains why long context and long outputs are charged more. Once you understand this, you can see why "trimming prompts" and "limiting output length" are the most direct ways to save money.

Reference Prices: A Roughly 4000× Gap Between the Most and Least Expensive

Below are approximate listed prices at the time of writing (per million tokens, USD). Always verify the provider's latest pricing page before actual budgeting.

Model	Input	Output
Amazon Nova Micro	$0.035	$0.14
GPT-4o Mini	$0.15	$0.60
DeepSeekAd-V3.2	$0.28	$0.42
Claude Haiku 4.5	$1.00	$5.00
Claude Sonnet 5.5	$2.00	$10.00
Claude Opus 5.5	$4.00	$20.00
o1 Pro	$150	$600

There is roughly a 4000× price gap between the cheapest and most expensive tiers. This means that model selection itself is the biggest cost lever you have — a wrong model choice can make your bill differ by three orders of magnitude, far exceeding any gains from prompt optimization.

Pricing mechanics of Prompt Caching and the Batch API: Prompt caching lets providers precompute and cache prefixes that repeat across multiple requests (such as fixed system prompts or product catalogs). Subsequent requests that hit the cache typically pay only about 10% of the original price, or even less. It's cheap because the attention computation on that content can be reused. The Batch API, meanwhile, queues non-urgent requests and processes them in bulk during idle compute periods, hence the roughly 50% price — at the cost of latency stretching from seconds to hours. The two don't conflict and can be stacked.

A Small Script: Run the Numbers

Rather than estimating by feel, it's better to plug in your numbers directly:

PRICES = {  # USD / per million tokens: (input, output)
    "nova-micro": (0.035, 0.14),
    "gpt-4o-mini": (0.15, 0.60),
    "haiku-4.5": (1.00, 5.00),
    "sonnet-5.5": (2.00, 10.00),
}

def monthly_cost(model, requests, in_tokens, out_tokens):
    p_in, p_out = PRICES[model]
    return requests * (in_tokens * p_in + out_tokens * p_out) / 1_000_000

# Natural language search: 100,000 queries per month, ~400 input / 100 output
for m in PRICES:
    print(f"{m:12s} ${monthly_cost(m, 100_000, 400, 100):,.2f}/month")

Taking "100,000 natural language searches per month, with roughly 400 input tokens and 100 output tokens" as an example, the script outputs:

nova-micro   $2.80/month
gpt-4o-mini  $12.00/month
haiku-4.5    $90.00/month
sonnet-5.5   $180.00/month

For the same workload, the model choice creates a price difference of more than 60×. This isn't theoretical speculation — it's a decision basis you can directly substitute with your own business numbers.

Bills from Two Real Scenarios

Scenario 1: Automatic product classification on an e-commerce platform

10,000 new products per day, each with roughly 200 input tokens (title + description + category list) and 30 output tokens. Using Claude Haiku 4.5:

• Input: 2 million tokens per day → $2.00
• Output: 300,000 tokens per day → $1.50
• Total roughly $3.50/day, about $105/month

If you put the category taxonomy into the cached prompt prefix, the input-side cost drops significantly. For tasks with "clear classification boundaries" like this, a cheaper model can often meet the quality requirements.

Scenario 2: Natural language search

100,000 searches per month at ~400 input / 100 output: Haiku is about $90/month, Sonnet about $180/month; if the quality is acceptable, Nova Micro can do it for under $3 per month.

The cost difference of the same feature across different models almost determines whether that feature can be launched at scale.

Five Practical Ways to Cut Costs

1. Match the model to the task scale. Tasks like classification, extraction, and routing rarely need a frontier model.
2. Trim your prompts. Lengthy system prompts are billed on every single call, which adds up to a considerable expense over time.
3. Cache reusable context. Product catalogs, policy documents, and style guides are all good candidates for the cached prefix.
4. Batch non-real-time tasks. Tasks like overnight enrichment and offline analysis don't need low latency, and the Batch API saves about half.
5. Model cascade. Let cheap models handle the easy 80%, and escalate only low-confidence cases to a stronger model.

What these five have in common: they optimize the invocation structure rather than sacrificing results. The real waste usually comes from "using the strongest model to do the simplest thing."

How a Model Cascade Works: The core of cascading is dividing labor across models at different price points. First, a cheap model handles all requests and outputs a confidence score or structured marker; only "hard cases" below the threshold are forwarded to a stronger model. This way, 80% of simple requests are handled by the cheap model, and only a few requests bear the high unit price. The benefit of cascading depends on the threshold setting: too loose a threshold lets quality problems slip through, while too tight a one pushes a large number of requests to the expensive model, defeating the purpose. In practice, it's often paired with small-sample evaluations to calibrate the threshold — the most cost-effective strategy, but also the one that needs the most tuning.

Are Open-Source Models Really "Free"?

Open-source models like Llama and Mistral have no per-token cost, but you pay for GPUs, hosting, monitoring, and engineering time. Until you reach a fairly high volume of calls, the all-in cost of a hosted API is usually lower.

In other words, "free" just moves the bill from the API ledger to the infrastructure and labor ledger. Only when call volume is large enough and the team has the corresponding operational capability does self-hosting come out ahead on total cost.

Final Thoughts

Controlling LLM costs can be boiled down to one action: measure the average input/output tokens per request, multiply by call volume, then start testing with the cheapest model that meets your quality requirements.

If your AI bill is growing faster than your usage, it often means there's room for optimization in model selection, caching, or routing strategy — worth a separate review.