Verified75% confidenceFactExact time
The inference stage of large models is memory-bound (limited by memory bandwidth); generating each token requires reading the entire model weights from VRAM, resulting in extremely low computational intensity.
5
Sources
75%
Confidence
Long-term
Relevance
8/23/2026
First Seen
Sources
Related Entities
Related Claims
Unverified在自回归生成阶段每生成一个token都需要从内存读取整个模型全部权重，使得生成阶段严重受限于内存带宽而非计算能力（memory-bound）79% similarPartially VerifiedLarge models are essentially stateless; during each inference, the model can only see the token sequence in the current input and has no persistent memory storage.78% similarUnverifiedThe memory of LLMs depends on the context window, which is the maximum number of tokens the model can process in a single inference.75% similarUnverifiedIn large model inference, generating each token requires reading all model weights from video memory, meaning the speed bottleneck is fundamentally a data movement problem rather than a computation problem.75% similarVerified自回归大模型推理瓶颈并非计算量而是内存带宽，推测解码通过拒绝采样保证输出分布与原始大模型完全一致74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/789729API
curl https://kongchang.com/api/v1/knowledge/claims/789729MCP
get_claim(id=789729)