Unverified50% confidenceFactExact time
自回归解码阶段每生成一个新token都需读取完整KV Cache，使解码阶段几乎总是受限于内存带宽而非计算能力
1
Sources
50%
Confidence
Long-term
Relevance
8/26/2026
First Seen
Sources
Related Entities
Related Claims
Unverified自回归生成中每生成一个token需读取全部权重一次，内存带宽而非算力峰值才是真正的性能瓶颈69% similarUnverified上下文越长，模型需要缓存的 KV Cache 越大，KV Cache 使模型生成新 Token 时无需重新计算所有历史 Token 的注意力权重69% similarUnverifiedThe LLM decoding process is memory bound, with the GPU spending most of its time waiting for data transfers during token-by-token generation.67% similarUnverified自回归推理的decode阶段是纯粹的内存带宽受限操作，将模型从FP16压缩到1-bit理论上可将token生成速度提升16倍66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/802107API
curl https://kongchang.com/api/v1/knowledge/claims/802107MCP
get_claim(id=802107)