The Truth About Prompt Cache Invalidation: A Misplaced Timestamp Costs You Full Price on Every Call

A Performance Problem That Never Throws an Error

Your prompt is long, and most of it is fixed: four thousand tokens of instructions, a product manual, a dozen examples. The user's actual question sits at the very bottom.

Your provider offers prompt caching. Someone turned it on. But the bill hasn't budged, and neither has latency. The dashboard shows that almost zero tokens are being cached.

No errors, no alerts — you're just paying full price every time. The problem lives in the first line of the prompt, where the current date and time are written out, down to the second.

That line was added for good reason: the model needs to know what day it is. But it was placed in the worst possible spot.

Prompt Caching Only Cares About Prefixes, Not Similarity

To understand this pitfall, you first need to understand how caching works.

The provider stores the processed result of the beginning of your prompt. A follow-up request can only reuse that cache if its prefix tokens match exactly. Note: not "similar" — from the very first token all the way to the cached point, it has to be identical.

If even a single character differs at the start, everything after it gets reprocessed as brand-new content and billed at full price.

The timestamp in the first line makes every request differ from the very first character. The prefix stored in the cache is never sent again. So the cache might as well not be enabled at all.

The Same Problem in More Subtle Forms

The timestamp is the most obvious offender, but it isn't the only one.

A username baked into the system prompt that differs per user; a tool list generated from a dictionary whose iteration order shifts between runs; a request id someone added for debugging; randomly selected examples meant to show the model some variety — all of these quietly break prefix stability, and none are as easy to spot as a timestamp.

What they share: they sit near the front of the prompt, yet change on every call.

What a token is: Large language models don't process characters or words directly. They split text into smaller units called tokens. A single English word might map to one token, or be broken into two or three; a Chinese character often maps to one to two tokens. Billing, context length limits, and cache-hit determination are all measured in tokens. The "four thousand tokens" mentioned in this article corresponds to roughly three thousand English words.

The granularity of prefix caching: Many providers (Anthropic, OpenAI, Google) match prompt caching at the level of token sequences, not strings. This means that even if two passages look identical to the human eye, the cache won't hit if their underlying tokenization differs — for example, from subtle differences in spaces, line breaks, or encoding. So the rule "it must be identical starting from the first token" is even stricter than it sounds at face value; even small formatting choices can invalidate the entire cached block.

Reorder Your Prompt by "How Often It Changes"

The solution proposed by author Serguey Asael Shinder is refreshingly plain: order your prompt by how frequently each piece of content changes.

Content that never changes goes first: instructions, fixed-order tool definitions, reference documents, examples.

Content that changes per session goes in the middle.

Content that changes per request goes last: the date, the user, the question itself.

The timestamp should be moved to the bottom, right next to the question it describes. That keeps the prefix stable, and only then does the cache actually take effect.

Verify With the usage Field, Not Your Gut

After you make the change, look at the usage block in every response. It tells you how many input tokens came from cache. Track that number alongside your costs.

The diagnostic is straightforward: if the prompt is mostly repeated content while cached tokens are near zero, something near the top of the prompt is moving. Follow that thread and you'll usually find the offending dynamic content pretty quickly.

Why session-level content belongs in the middle: Session-level content (e.g., multi-turn conversation history, the current task's context) is stable within a single user session but changes across sessions. Placing it after the stable prefix lets you keep reusing the first half of the cache across multiple requests within the same session; once the session ends, the new prefix starts stable from scratch again, untainted by the previous session.

Cache TTL and hit conditions: Most providers' caches have a time-to-live (usually a few minutes to an hour) and require explicitly marked cache breakpoints. Even with a perfectly identical prefix, if two requests are separated by more than the TTL, or the cache marker wasn't placed at the right spot, you still won't get a hit. When diagnosing "cache invalidation," aside from checking whether the prefix is stable, also confirm these two configurations are correct.

What the usage field typically contains: Mainstream APIs return a usage object in their responses, commonly with several fields: input_tokens (total input tokens this call), output_tokens (generated tokens), cache_creation_input_tokens (tokens written to cache this call), and cache_read_input_tokens (tokens read from cache this call). Naming varies slightly across providers, but the semantics are similar.

Turning numbers into a diagnosis: Compare the ratio of cache_read_input_tokens to input_tokens. If the prompt is overwhelmingly fixed content but cache reads stay near zero over time, you can be almost certain the prefix is changing between requests. Conversely, if the ratio holds steady at a high level, the cache is working. Log these two numbers side by side with the actual cost per call, and you can pinpoint when the problem started without even touching the code.

An Engineering Principle That's Easy to Overlook

The value here isn't just a cost-saving trick. It reveals a broader engineering rule: the arrangement of a system should reflect how often each part changes.

Caching only helps when you send the same content in the same order twice. Put the stable parts first and keep them still — this principle applies to prompt caching, and to any optimization mechanism based on prefix matching.

One line added so "the model knows today's date" costs you four thousand tokens reprocessed at full price on every call. The fix might take just a few minutes — but only if you first realize the cache isn't working at all.