Unverified50% confidenceFactExact time
The KV cache grows linearly with context length, increasing the memory read volume per step in the decoding phase, which is the fundamental reason decoding speed decreases as context grows.
1
Sources
50%
Confidence
Long-term
Relevance
10/2/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedKV Cache将已计算的历史键值矩阵缓存在内存中，将自回归生成的复杂度从随序列长度平方增长降至线性78% similarVerifiedThe KV Cache (key-value cache) mechanism accelerates inference by caching the Key and Value vectors generated by the self-attention layer to avoid redundant computation, but the longer the context, the more vectors must reside in VRAM, causing memory pressure to grow in a near-linear fashion.78% similarUnverified在推理阶段内存带宽往往比纯算力更早成为瓶颈，注意力机制需在每步生成时读取全部KV缓存，上下文长度增大时内存读写开销呈平方级增长73% similarUnverifiedKVarN在量化前对每个通道进行方差归一化（除以标准差），使各通道数值落入相似范围后再进行均匀量化，反量化时乘回标准差恢复71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/968808API
curl https://kongchang.com/api/v1/knowledge/claims/968808MCP
get_claim(id=968808)