Temperature and Seed: The Hidden Switch That Doubles Your LLM Extraction Costs

The same batch of 100,000 JSON extraction tasks could cost you $667 or $1,100 — and the per-token price hasn't moved a cent. The difference isn't on the vendor's price list; it's in those two parameters you copied from a tutorial and never touched again: temperature and seed.

Why the per-call price stays the same while the bill doubles

Let's clarify a few basics first. A token is roughly ¾ of a word, APIs bill per million tokens, and input and output are priced separately. Temperature controls how adventurous the model is when picking the next token: 0 means it almost always picks the highest-probability one, while 1 and above spreads the bets wide. Seed is a number that makes random choices reproducible — same seed, same input, usually the same answer. Validation is your code's check before using the answer: does it parse as JSON? Are all required fields present?

Neither knob shows up on the invoice. The same request at temperature 0 and temperature 1.2 is billed identically — input tokens times input price, plus output tokens times output price.

Take a concrete task: extract five fields from a customer support ticket and return them as JSON. Assume 1,500 input tokens (instructions plus ticket) and 300 output tokens. At GPT-5.6's official pricing of $2 per million input and $12 per million output:

1,500 × $2 / 1M + 300 × $12 / 1M = $0.0030 + $0.0036 = $0.0066 per call.

That number holds at any temperature. What changes is how many times you pay it.

How much a single failed answer actually costs you

The API returned 200 OK, you got billed, but the answer looks like this: {"priority": "high", "summary": "Customer says "refund" ... — an unescaped quote, a trailing comment, or a friendly "Sure! Here's your JSON:" up front. The parser throws, your code retries, and you pay again.

High temperatures raise the probability of this failure for a simple reason: valid JSON is a narrow path, and every sample that drifts from the most likely token is a chance to step off it. Let p be the fraction of answers that pass validation. If you keep retrying until one passes, the expected number of attempts is 1/p, so the cost to complete one task is per-call price ÷ p.

The original article's code example makes this clear:

CALL = 1_500 * 2.00 / 1e6 + 300 * 12.00 / 1e6   # $0.0066, GPT-5.6 official
TASKS = 100_000

for p in (0.99, 0.90, 0.75, 0.60):
    attempts = 1 / p
    print(f"p={p:.2f}  attempts={attempts:.2f}  bill=${CALL / p * TASKS:,.0f}")

The results are in the table below:

Pass rate p	Attempts per task	Total bill for 100K tasks
0.99	1.01	$667
0.90	1.11	$733
0.75	1.33	$880
0.60	1.67	$1,100

To be clear: these p values are illustrative, not measured. The original author did not run a temperature sweep for this article; your p depends on your model, prompt, and schema. The arithmetic is exact, but the inputs are model-set. Log every validation failure alongside the temperature in use, and within a day you'll have your own p.

Deriving the expected number of attempts: Let p be the probability that any single call passes validation, with calls independent of each other. Then the number of attempts until the first success follows a geometric distribution, whose expected value is 1/p. This is the classic geometric-distribution result — for example, at p=0.75 the expected number of attempts is 1.33, meaning each task needs 1.33 billed calls on average. There's an implicit assumption here: every retry is an independent, identically distributed sample. If seed and temperature stay exactly the same on retry, the independence assumption breaks, the model reproduces the same output, and the 1/p formula no longer holds — costs can trend toward infinity. So the arithmetic of "per-call price divided by p" is only reliable when each attempt genuinely introduces new randomness.

Set the seed wrong and your retries may be entirely wasted

Here's a trap that turns 1/p into infinity. If you fixed the seed for reproducibility — a good habit in testing — and then retry with the same seed, same temperature, and same prompt, you're asking the model to repeat itself. It will oblige, broken comma and all. Every retry is billed, but none gets a new chance to pass. (Seed determinism is "best effort," so you might get lucky, but don't build on that luck.)

So retries must change something. Drop or adjust the seed, lower the temperature on retry, or append the parse error to the prompt so the model knows what to fix. The last approach adds input tokens to each retry — those 300 bad-answer tokens plus the error message — so retries cost slightly more than the first call, making the table above a floor rather than a ceiling.

Also cap the loop. With a maximum of 3 attempts and p=0.75, the average is 1.31 attempts instead of 1.33, but 1.6% of tasks (0.25³) will fail permanently — you'd rather know that than pay for an eleventh attempt.

Structured output is a constrained-decoding mechanism offered by mainstream LLM vendors. Plain JSON mode only guarantees the output is valid JSON, not that its fields match your schema; structured output lets you attach a JSON Schema to the request (field names, types, enum values, required fields), and during inference the model is constrained to generate only token sequences that conform to that schema. The usual implementation masks tokens that violate the schema during decoding (grammar-constrained decoding), so in theory it can push the pass rate of the "does it parse" portion close to 1. It can't guarantee the semantic correctness of field values, nor cover constraints the schema doesn't declare, but it's extremely effective at eliminating format failures. Before using it, confirm your model and API version support it, and note the slight latency overhead the first time a schema is used.

Three things you can do Monday morning

Set temperature to 0 for any output a parser needs to read. Then, if your vendor supports it, use structured output mode — the request carries a JSON schema, and the model is constrained to produce text matching it, pushing the "does it parse" portion of p toward 1. Finally, start counting. A simple counter of validation_failures / calls is your p, and price ÷ p is what you actually pay per task.

The original article also mentions another cost-optimization lever: the same GPT-5.6 at official pricing of $2 per million input and $12 per million output can go through the batch channel at 1.70/10.20 — a 15% discount. At p=0.60, that turns $1,100 into $935. Fixing p to 0.99 at official pricing gets you to $667. A discount just multiplies your bill by a fixed number, whereas 1/p can multiply by any number. Do the cheap thing first.

The Batch API is an asynchronous discount channel offered by several vendors: you bundle a large number of requests into a file and submit it, the vendor processes them within a committed time window (e.g., 24 hours), and the price is typically around 50% of the synchronous endpoint — the 15% discount on GPT-5.6 mentioned in this article falls into this category. It suits offline, non-realtime, delay-tolerant tasks, such as this article's 100,000 JSON extractions — no user is waiting in the foreground, so they can absolutely go through the batch channel. Note that a batch discount is a fixed coefficient multiplied onto the unit price; it doesn't change the validation pass rate p, so it's orthogonal to lowering temperature and enabling structured output: the former shrinks the per-call price, the latter reduces the number of calls you need to bill. When combined, total cost is per-call price × discount coefficient ÷ p × number of tasks.

What this method can't solve

Valid JSON doesn't mean correct JSON. The schema can guarantee the priority field exists, but it can't stop the model from filling in the wrong priority — and those failures don't trigger retries at all; they slip straight into your database. Low temperature isn't right for every task either: during brainstorming it produces flat, repetitive text, and there's nothing to validate there anyway. Network errors and timeouts are a separate retry budget with its own arithmetic.

Go dig up the temperature setting in your extraction code. If it says 0.7 because some quick-start tutorial said so, you now know where part of your bill is going.

One-line Summary

Temperature and seed don't change the unit price, but through the validation pass rate p they can flip a 100,000-task bill from $667 to $1,100.

Paragraph Summary

The article points out that the same batch of 100,000 JSON extraction tasks can cost anywhere from $667 to $1,100 while the per-token price stays the same — the difference comes from the often-overlooked temperature and seed parameters in your code. Temperature controls sampling randomness, and high temperatures raise the chance that output deviates from valid JSON; every validation failure requires a retry and another billed call, so the real cost of completing a task is the per-call price divided by the validation pass rate p. The article uses the geometric distribution to derive an expected attempt count of 1/p, and presents an illustrative table showing the bill rising from $667 to $1,100 as p drops from 0.99 to 0.60. A more insidious trap is that retrying with a fixed seed makes the model reproduce the same error, so you pay without passing validation. The author recommends setting temperature to 0, enabling structured output, and logging validation_failures/calls to measure p, noting that batch discounts are just a fixed multiplier while fixing p is the more effective lever.