Unverified60% confidenceFactExact time
Generation throughput (tok/s) reflects decoding-phase efficiency and is affected by KV Cache size and memory bandwidth, with far lower sensitivity to input length than prefilling.
2
Sources
60%
Confidence
Long-term
Relevance
9/17/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe size of the KV cache grows linearly with context length, and VRAM consumption increases significantly when processing longer texts.77% similarUnverified自回归解码阶段每生成一个新token都需读取完整KV Cache，使解码阶段几乎总是受限于内存带宽而非计算能力74% similarUnverifiedKV cache runtime provides a low-cost, fine-grained, real-time intervention capability that fills the gap between model fine-tuning and external scaffolding.74% similarVerifiedKV Cache reduces the computational complexity of self-attention from O(N²) to O(N) by caching already-computed Key-Value pairs in GPU memory, but memory usage grows linearly with sequence length.73% similarPartially VerifiedKV Cache avoids redundant computation by caching already-computed Key and Value matrices, and advanced techniques such as PagedAttention further optimize cache memory management efficiency.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/930282API
curl https://kongchang.com/api/v1/knowledge/claims/930282MCP
get_claim(id=930282)