Unverified50% confidenceFactExact time
A prefix tree (Trie) organizes cached KV into a tree by token sequence, so multiple requests can hit and reuse KV as long as they share a common prefix of any length, without requiring the entire prefix to be identical.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedA prefix-shared trajectory tree allows the KV cache and attention computation of the forward pass to be reused on the shared prefix, reducing VRAM and compute overhead under long contexts.69% similarVerifiedThe KV Cache mechanism can cache repeated prefix content, and the more concise the system prompt, the higher the cache hit rate.67% similarUnverifiedThe Transformer's causal attention mechanism dictates that each token's KV is related only to all the tokens before it, so prompts with completely identical prefixes generate completely identical KV that can be reused directly.65% similarUnverifiedIn bidirectional attention models (such as BERT encoders), each token's representation is affected by the entire text, so prefix KV cannot be reused independently; therefore Prompt Caching mainly appears on generative large models rather than encoder models.63% similarUnverifiedKV Cache avoids redundant computation during autoregressive inference by caching the K and V values of historical tokens, so that new tokens only need to compute their own Q.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1010942API
curl https://kongchang.com/api/v1/knowledge/claims/1010942MCP
get_claim(id=1010942)