The "Friendly and Professional" Line in Your System Prompt Costs $708 a Year

Every chatbot tutorial starts the same way: write a system prompt. You write one, a teammate adds a tone requirement, someone pastes in the company history, and three months later the prompt looks more like a README.

The problem is that nobody ever put a price on that README. Because it never shows up as a separate line item—it's just input tokens on every call, forever.

A token is roughly ¾ of an English word, and APIs bill per million tokens. The system prompt is the block of instructions sent before the user's message, and the API is stateless—it doesn't remember any of it between calls. You resend it on every request, and you pay for it every time.

What Real Traffic Looks Like

Start with some measured data from a gateway: across 5,546 successful chat calls from 16 customer accounts (over the period described in the original article), the median request carried 1,280 input tokens and returned 108 output tokens. Requests are about 12 times larger than responses, and in a typical bot, most of that input is the same fixed block of instruction text, sent over and over. That's what a system prompt looks like on the bill.

Doing the Math on a Typical Prompt

The author wrote a realistic support-bot prompt for a fictional electronics store: role definition, return policy, answer format, escalation rules, four few-shot examples (sample conversations showing the desired style), and a line with today's date. Then counted it with tiktoken and o200k_base (the tokenizer behind current GPT models).

The full prompt comes to 946 tokens. Policies account for 234, examples for 192, and answer format for 127. One passage alone takes up 97 tokens: "Remember that you are a helpful, friendly, and professional assistant. Always do your best to give accurate, complete, well-structured answers..."—eighty-one words that every modern model does by default anyway.

The Price of One Paragraph

Here's the annual cost of those 97 tokens at official input pricing, depending on the model and daily call volume:

Model (input $/M)	1K/day	10K/day	100K/day
Gemini 3.1 Flash Lite ($0.25)	$9	$89	$885
Claude Sonnet 5.5 ($2.00)	$71	$708	$7,081
GPT-6 Astra ($10.00)	$354	$3,541	$35,405

The formula is a single line: tokens × calls_per_day × 365 / 1,000,000 × input_price. On a cheap model, this paragraph is a rounding error. On a mid-tier model at moderate traffic, it's a monthly subscription you don't need. On a flagship model at scale, it's a salary.

Some will say "input is cheap, output is what's expensive." Per-token, yes—but not per request: with a 1,280-token request and a 108-token response, total input cost still exceeds output even if output is five times pricier per token.

Few-shot examples are pairs of "user question—ideal answer" embedded in the prompt to demonstrate the desired answer style, format, and reasoning pattern to the model. It works like in-context learning without fine-tuning the model: the model infers the task pattern from the examples and applies it to real user messages. Few-shot examples are often highly effective, but the cost is that each one permanently occupies input tokens—four short examples can easily hit around 200 tokens, creating a substantial and ongoing cost at high traffic. When the examples merely restate common-sense style the model already knows, they're often candidates for removal.

Auditing the Prompt: Cut First, Then Cache

Go through the prompt section by section and ask one question: if I delete this, will the answer change?

Cut what the model already does or never uses. In the example, that's the "helpful" passage (97 tokens), the company history and mission statement (106 tokens—the bot doesn't need to know the founding year to process returns), and two rules repeated verbatim in the format section (23 tokens). That's 226 tokens total, 24% of the prompt, or about $1,650 a year on Sonnet 5.5 at 10K calls a day. After cutting, run your eval set; if quality is unchanged, those tokens were just decoration.

Cache the long, stable content. Prompt caching is when a provider retains the beginning of a request and bills repeated reads at a fraction of the normal input price. The catch: it only works on byte-for-byte identical prefixes. The original prompt put today's date and the customer's name at the top—one changing line makes the whole prompt uncacheable. Move all dynamic content to the end:

messages = [
    {"role": "system", "content": ROLE + POLICIES + FORMAT + EXAMPLES},  # never changes
    {"role": "system", "content": f"Today: {today}. Customer: {name}."}, # changes
    {"role": "user",   "content": user_message},
]

Cutting and caching aren't either/or. Cut first, because deleted tokens cost zero, while cached tokens still cost a little on every call—and full price when the cache expires.

Prompt caching deserves a closer look at how it works: the provider keeps a key-value cache (KV cache) of the request prefix on the server side, and when a subsequent request's beginning matches a previous one byte for byte, it reuses the already-computed state instead of reprocessing those tokens. Billing is therefore typically split into "cache write" (slightly above normal input price) and "cache read" (significantly below normal input price, commonly around one-tenth). The hit condition is very strict—any single character change in the prefix invalidates the cache, which is why dynamic content like dates and usernames must be moved to the end of the prompt. In addition, cache entries usually have a time-to-live (e.g., 5 minutes to 1 hour), so with sparse traffic the cache tends to expire, hit rates drop, and the write premium can't be amortized—making caching potentially uneconomical in low-frequency scenarios.

What This Doesn't Solve

Caching has a break-even point: writes cost more than normal input, so at low hit rates it can actually raise your bill. Some providers only cache prefixes above a minimum length—check the docs before relying on it. Token counts here come from OpenAI's tokenizer; Anthropic and Google tokenize differently, so the numbers will be off by a few percentage points. And the gateway's own usage records don't yet break out cached tokens, so real cache hit rates can't be shown.

The prices above are the vendors' official rates. The gateway resells the same models below official pricing—Claude Sonnet 5.5 at $1.70 per million input tokens versus the official $2.00—which brings that $708 paragraph down to $602. That's a fixed percentage discount, and it doesn't delete the paragraph. Discounting tokens you don't need is still paying for tokens you don't need.

Open your system prompt, run a script over it, and find the most expensive passage. It's probably the one nobody remembers writing.

One-line Summary

System prompts are billed again on every request; trim the redundancy, then cache the stable prefix to save significantly.

Paragraph Summary

The article points out that because APIs are stateless, system prompts are resent and re-billed on every request, yet are rarely examined as a standalone cost item. Based on data from 5,546 real calls, the median request carries 1,280 input tokens and returns only 108 output tokens—making input cost exceed output on a per-request basis. Using a 946-token e-commerce support prompt as an example, a 97-token "helpful" instruction alone costs $708 a year on Claude Sonnet 5.5 at 10K calls a day. The fix comes in two steps: first audit section by section and cut what the model does by default or never uses (24% reducible in the example), then move dynamic content to the end to enable prompt caching. The article also notes limitations such as the cache break-even point, minimum length requirements, and tokenizer differences.