ConceptCausal Attention
Causal Attention
The core self-attention mechanism of the Transformer decoder, which uses a lower-triangular mask to ensure that the token at position i can only attend to the prefix from 0 to i, forming the theoretical basis for token-by-token generation and prefix KV reuse.
Timeline (last 90 days)
Oct 12
The Transformer's causal attention mechanism dictates that each token's KV is related only to all the tokens before it, so prompts with completely identical prefixes generate completely identical KV that can be reused directly.
Unverified50%