Unverified60% confidenceFactExact time
In autoregressive inference, the memory footprint of the KV cache grows linearly with sequence length and may equal or exceed the weight volume in long-context scenarios.
2
Sources
60%
Confidence
Long-term
Relevance
8/31/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe GPU memory occupied by the KV cache grows with context length and dialogue turns, and is the main source of memory usage and latency in long-context deployment.78% similarUnverifiedKV cache 的显存占用随上下文长度线性增长，且每个并发请求、每条序列都维护一份独立缓存。76% similarUnverifiedFor long-context requests, the KV cache data volume can reach several GB76% similarUnverifiedThe KV cache grows continuously with sequence length, delivering powerful long-range retrieval capability at the cost of memory that cannot be digested into the model itself.75% similarUnverifiedWhen large models generate long texts, they need to cache the key-value pairs (KV Cache) of all previous tokens, and the memory footprint grows linearly with context length, making it very easy for this to become a memory bottleneck in high-concurrency scenarios.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/830173API
curl https://kongchang.com/api/v1/knowledge/claims/830173MCP
get_claim(id=830173)