Autoregression
A generation mechanism used in language models whereby each step predicts the next token conditioned on all previously generated prefix tokens, incrementally constructing a complete sequence. Autoregression is the core mechanism of generative large language models such as GPT.
Core Facts
Timeline (last 90 days)
A classifier maps input to one of a finite set of predefined categories with a fixed, closed candidate space; a generative language model predicts the next token from the entire vocabulary at each step, with an open candidate space, building output autoregressively.
Autoregressive generation is the core decoding paradigm of current mainstream large language models (such as the GPT series): generating one token at a time and appending it to the end of the sequence, looping until an end token is encountered.
The formula for local AI inference speed is Token/s ≈ memory bandwidth / data volume to be read per token, and large language models generate each token in an autoregressive manner.
Due to the autoregressive nature, each decode step requires accessing the KV Cache of all previous tokens.
LLMs generate text in an autoregressive manner, producing only one token at a time, and the generation of the next token must wait for the previous token to be completed.
The autoregressive token prediction paradigm is built on correlation rather than causality.
自回归推理时batch size通常为1，每个token生成只做一次矩阵-向量乘法，算术强度极低，导致memory-bound问题，单用户场景下GPU利用率往往不到10%
现代主流LLM采用自回归生成方式，逐个token按顺序生成
自回归推理是memory-bound（内存带宽受限）问题，因为batch size通常为1时算术强度极低，GPU大部分时间在等待数据搬运
Large model output generation uses an autoregressive approach to produce tokens one by one, requiring recalculation of attention over all preceding context with each token generated.
2 more timeline events
All Facts (12)
Large model output generation uses an autoregressive approach to produce tokens one by one, requiring recalculation of attention over all preceding context with each token generated.
65%UnverifiedLLMs generate text in an autoregressive manner, producing only one token at a time, and the generation of the next token must wait for the previous token to be completed.
70%UnverifiedModern large language models generate code in an auto-regressive manner, predicting the next token each time by referencing only the existing context.
60%UnverifiedA classifier maps input to one of a finite set of predefined categories with a fixed, closed candidate space; a generative language model predicts the next token from the entire vocabulary at each step, with an open candidate space, building output autoregressively.
50%UnverifiedAutoregressive generation is the core decoding paradigm of current mainstream large language models (such as the GPT series): generating one token at a time and appending it to the end of the sequence, looping until an end token is encountered.
50%UnverifiedThe formula for local AI inference speed is Token/s ≈ memory bandwidth / data volume to be read per token, and large language models generate each token in an autoregressive manner.
50%UnverifiedDue to the autoregressive nature, each decode step requires accessing the KV Cache of all previous tokens.
50%UnverifiedThe autoregressive token prediction paradigm is built on correlation rather than causality.
50%Unverified现代主流LLM采用自回归生成方式，逐个token按顺序生成
50%Unverified自回归推理时batch size通常为1，每个token生成只做一次矩阵-向量乘法，算术强度极低，导致memory-bound问题，单用户场景下GPU利用率往往不到10%
50%Unverified自回归推理是memory-bound（内存带宽受限）问题，因为batch size通常为1时算术强度极低，GPU大部分时间在等待数据搬运
50%Unverified输出生成是严格串行的自回归过程，每生成一个token都需要一次完整的模型前向传播
50%