Unverified50% confidenceFactExact time
KV cache size grows linearly with context length, and attention computation grows quadratically with context length.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe KV cache grows linearly with context length, increasing the memory read volume per step in the decoding phase, which is the fundamental reason decoding speed decreases as context grows.80% similarUnverifiedFor multi-turn conversations or long-document question answering, if the KV of the entire context is recomputed on every request, the compute overhead grows quadratically with context length.79% similarVerifiedThe KV Cache (key-value cache) mechanism accelerates inference by caching the Key and Value vectors generated by the self-attention layer to avoid redundant computation, but the longer the context, the more vectors must reside in VRAM, causing memory pressure to grow in a near-linear fashion.77% similarUnverifiedStandard attention's computation grows quadratically with sequence length, whereas the low-rank bottleneck decomposes the transformation into two steps: first compressing to r dimensions and then expanding.77% similarUnverifiedThe KV Cache footprint of linear attention is independent of sequence length, whereas the cache of traditional attention grows linearly with sequence length.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1037892API
curl https://kongchang.com/api/v1/knowledge/claims/1037892MCP
get_claim(id=1037892)