Unverified50% confidenceFactExact time
For autoregressive language models, generating k tokens requires k serial forward passes, so latency grows linearly with output length; whereas a single-forward-pass decision model has a fixed output length, and its latency is essentially independent of the number of options.
1
Sources
50%
Confidence
Long-term
Relevance
11/2/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAutoregressive models can only determine one token per forward pass, so inference cannot be parallelized along the time dimension, and latency grows linearly with output length.73% similarUnverifiedLLMs generate text in an autoregressive manner, producing only one token at a time, requiring N forward inference passes to generate N tokens, with latency scaling linearly with output length.73% similarUnverified大模型推理的自回归生成需要逐Token串行输出，生成N个Token需要N次前向传播72% similarVerifiedThe autoregressive generation process of reasoning models is strictly sequential; KV Cache VRAM usage grows linearly with the length of the thinking token sequence, making memory bandwidth the performance bottleneck.72% similarUnverifiedWithout KV Cache, the model would need to recompute all preceding content every time a new token is generated, with computational complexity growing quadratically with sequence length.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1125169API
curl https://kongchang.com/api/v1/knowledge/claims/1125169MCP
get_claim(id=1125169)