The Real Cost of LLM Calls: Tokens, Precision, and Honest Numbers

Sticker Price ≠ Actual Cost

The per-million-token rate is a price tag, not a true cost. When two providers quote the same per-million-token price, the final bill can still differ dramatically — because the gap hides in four dimensions that most teams overlook.

The real cost of an LLM call depends on how many tokens you actually consume, what numerical precision the provider uses to serve the model, how long requests wait, and whether the endpoint stays alive under load. Quantify all of these clearly, and you can put a defensible price on a single call.

The Four Corners Where Hidden Costs Live

What separates expensive from cheap isn't the unit price — it's the variables the unit price ignores.

Token consumption. Reasoning models and verbose system prompts can dramatically change how many tokens a task actually burns. The unit price is fixed; the quantity consumed is not. A seemingly cheap model that emits large volumes of intermediate reasoning tokens can cost far more in total than a nominally pricier alternative.

Numerical precision. Models deployed at lower numerical precision cost less to run, and providers don't always tell you which precision you're getting. Precision is both a cost lever and a quality lever — it determines the provider's hardware overhead and influences model output behavior.

Latency. Time to first token and tokens per second directly determine whether a call fits inside a user-facing time budget. Public pricing pages almost never publish these numbers.

Availability. A cheap endpoint that returns 503 under heavy load has effectively infinite cost for every failed request. No unit price recovers a call that never completes.

Precision Deserves Its Own Line Item

When you compare two deployments of the same model, the startup flags in the command line don't tell the whole story. Even when launch configurations look identical, the actual precision of the weights serving traffic may differ — and that changes both speed and output quality.

When benchmarking, lock down the precision you're actually measuring and state it explicitly. A fair comparison names the precision; an unfair one buries it.

A simple way to stay honest is to treat precision as an explicit weight in your cost model rather than quietly folding it into a single price figure. Surface the assumption first, then let readers adjust it themselves.

---

Numerical Precision refers to the floating-point format used for model weights. Common formats include FP32 (32-bit full precision), FP16 (16-bit half precision), BF16 (Brain Float 16), and various quantized formats such as INT8 and INT4. Lower precision means each parameter occupies less GPU memory and requires less compute to run, reducing the provider's hardware costs and typically improving response speed. The trade-off is a narrower range and resolution of numerical representation, which can introduce subtle but reproducible quality degradation in fine-grained reasoning, arithmetic, or low-frequency vocabulary handling. A common industry approach is to quantize weights to INT4 or INT8 while preserving higher precision in critical layers, balancing cost against quality. Because quantization strategies vary widely and providers rarely disclose them proactively, external users can't reliably infer deployment precision from an API endpoint name alone — which is precisely why measuring both behavioral performance and speed on a fixed prompt is an important verification method.

---

Latency Is a Cost, Not a Feeling

Time to first token and tokens per second are measurable quantities. They belong in any cost narrative. The problem is that public APIs rarely expose these metrics, so you have to call the endpoint yourself with the same prompt and record the numbers.

One call is a sample, not a law. Report how many calls you made and where you made them from, because the answers change with both variables. Treating latency as something you judge by subjective "feel" is a failure of measurement discipline.

---

Time to First Token (TTFT) is the elapsed time from sending a request to receiving the first output token — it reflects the model's preprocessing and scheduling overhead before generation begins. Tokens per Second (TPS) measures the speed of subsequent streamed output and determines total wait time for long responses. Together they define the user-perceived "response rhythm": low TTFT gets the first character on screen quickly; high TPS fills in the content fast. For user-facing products, there is typically a hard time budget (e.g., first character within 200 ms, full response within 3 seconds). Exceeding that budget directly triggers timeouts, fallbacks, or user abandonment — at which point latency is no longer a "UX metric" but a business cost that determines whether a call succeeds. Measurement requires controlling for network origin (e.g., same data center vs. end-user device) and sampling across multiple time periods, because queuing latency on shared inference clusters fluctuates significantly with load.

---

An Actionable Measurement Checklist

Before you publish a cost figure for an LLM call, verify each of the following:

1. You counted actual token usage for a representative task — not just the unit price.
2. You declared the precision of the service you measured.
3. You personally measured time to first token and throughput, and noted the number of calls.
4. You checked availability under the load you actually care about, not during idle periods.
5. You separated measured numbers from assumed numbers and labeled the assumptions.

If a comparison is missing any of the above, it's a quote — not a measurement.

Why This Matters More Now

As more products route requests across multiple models, the question has shifted from "which model is best in the abstract" to "which deployment delivers the quality I need at a cost I can defend."

That's a measurement problem. Measurement problems reward discipline, not marketing. When selection moves from a single model to a cross-model portfolio, gut-feel "cheap" judgments get exposed by real invoices.

FAQ

Is per-million-token pricing useless? No — it's an input. Real cost = unit price × actual token count, adjusted for precision, latency, and availability.

If the provider won't disclose precision, how do I measure it? Measure model behavior and speed on a fixed prompt, and explicitly state "precision as served is unverified" when you can't confirm it.

Why measure latency yourself? Because most public APIs don't expose time to first token, and it's decisive for user-facing time budgets.

One call or many? Many. One call is a sample. Note the number of calls and the originating location.

Closing: A Number's Value Depends on What's Attached to It

The transferable principle here is this: a cost figure is only as valuable as the conditions attached to it. State the precision. Note the number of calls. Separate measured values from assumed ones — and the number becomes something you can actually defend.