Cache Key
A key value in a caching system that uniquely identifies a cache entry, typically composed of a combination of multiple fields. A cache hit is registered only when all fields match.
Timeline (last 90 days)
In the official example, a cache key is turned into a pin tumbler lock, which can only turn when all the pins match.
The judgment logic of a cache key is isomorphic to a pin tumbler lock: only when every field matches one by one is it considered a cache hit and the result returned.
Cache key design must include sufficient request identity information; using only prompt content or session ID as the key is prone to collisions under concurrency, so a request-level unique identifier should be introduced along with a strict invalidation policy.
The industry emphasizes that cache keys must be bound to a tenant or request identity, rather than being reused solely on the basis of content similarity.
Joining the three inputs with \u0000 in the cache key avoids field-boundary confusion (e.g., 'ab'+'c' and 'a'+'bc' producing the same string).
Caching verdicts under a key that is just a hash of the prompt text is itself a bug: the verdict is a function of the diff, the prompt template, and the answering model, so hashing only the diff causes stale text to be silently reused after upstream changes an identifier.
The final cache key that passed contains three things—modelId, templateHash, and diffHash—joined with \u0000 and then SHA-256 hashed.
The fix for the system prompt not entering the hash: explicitly assert that the first item in the message list must be the system role, and write the byte hash of the prompt file into the cache key.
Cache keys should include the base URL, model ID, complete messages array, temperature/top_p/max_tokens/stop/seed, tools/tool_choice/response_format, and a version string.
When serializing cache keys, key names must be sorted so that {"a":1,"b":2} and {"b":2,"a":1} hash to the same value.
All Facts (10)
In the official example, a cache key is turned into a pin tumbler lock, which can only turn when all the pins match.
50%UnverifiedThe judgment logic of a cache key is isomorphic to a pin tumbler lock: only when every field matches one by one is it considered a cache hit and the result returned.
50%UnverifiedCache key design must include sufficient request identity information; using only prompt content or session ID as the key is prone to collisions under concurrency, so a request-level unique identifier should be introduced along with a strict invalidation policy.
50%UnverifiedThe industry emphasizes that cache keys must be bound to a tenant or request identity, rather than being reused solely on the basis of content similarity.
50%UnverifiedJoining the three inputs with \u0000 in the cache key avoids field-boundary confusion (e.g., 'ab'+'c' and 'a'+'bc' producing the same string).
50%UnverifiedCaching verdicts under a key that is just a hash of the prompt text is itself a bug: the verdict is a function of the diff, the prompt template, and the answering model, so hashing only the diff causes stale text to be silently reused after upstream changes an identifier.
50%UnverifiedThe final cache key that passed contains three things—modelId, templateHash, and diffHash—joined with \u0000 and then SHA-256 hashed.
50%UnverifiedThe fix for the system prompt not entering the hash: explicitly assert that the first item in the message list must be the system role, and write the byte hash of the prompt file into the cache key.
50%UnverifiedCache keys should include the base URL, model ID, complete messages array, temperature/top_p/max_tokens/stop/seed, tools/tool_choice/response_format, and a version string.
50%UnverifiedWhen serializing cache keys, key names must be sorted so that {"a":1,"b":2} and {"b":2,"a":1} hash to the same value.
50%