Unverified50% confidenceFactExact time
The Transformer's causal attention mechanism dictates that each token's KV is related only to all the tokens before it, so prompts with completely identical prefixes generate completely identical KV that can be reused directly.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn bidirectional attention models (such as BERT encoders), each token's representation is affected by the entire text, so prefix KV cannot be reused independently; therefore Prompt Caching mainly appears on generative large models rather than encoder models.71% similarUnverifiedKV Cache avoids redundant computation during autoregressive inference by caching the K and V values of historical tokens, so that new tokens only need to compute their own Q.67% similarVerifiedThe KV Cache mechanism can cache repeated prefix content, and the more concise the system prompt, the higher the cache hit rate.66% similarUnverifiedA prefix tree (Trie) organizes cached KV into a tree by token sequence, so multiple requests can hit and reuse KV as long as they share a common prefix of any length, without requiring the entire prefix to be identical.65% similarUnverifiedWhen KV cache is reused, cached intermediate activations may produce different logit outputs for the same input when the batch size or sequence concatenation method changes.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1010933API
curl https://kongchang.com/api/v1/knowledge/claims/1010933MCP
get_claim(id=1010933)