Prefill

The Prefill phase is a compute-intensive operation, while the Decode phase is a memory-bandwidth-intensive operation.

The Prefill phase is a compute-intensive task with high demands on compute power; the Decode phase is a memory-access-intensive task that is sensitive to GPU memory bandwidth.

The prefill stage of Transformer inference is primarily bottlenecked by compute (FLOPS), while the decode stage is primarily bottlenecked by memory bandwidth.

TTFT is determined by the Prefill phase and is compute-bound, while TPOT corresponds to the Decode phase and is memory-bandwidth-bound.

传统混合部署中Prefill阶段的大批量计算会阻塞Decode阶段的token生成，导致TTFT波动剧烈

LLM 推理分为预填充阶段（Prefill，计算密集型）和解码阶段（Decode，内存带宽密集型）两个阶段

Large model inference is divided into two phases, Prefill and Decode; Prefill is a major component of Time to First Token (TTFT), while Decode corresponds to Time Per Output Token (TPOT).

The prefill phase processes the complete input context, computes attention for all input tokens, and generates the KV cache; it is a compute-intensive operation whose time consumption is strongly correlated with input length.

Disaggregated inference decouples the compute-intensive prefill stage from the memory-bandwidth-intensive decode stage, allocating them to separate hardware resource pools best suited to each for independent scaling.

The prefill stage is highly parallel with large throughput but latency that grows with context length, while the decode stage generates tokens auto-regressively and its speed is limited by memory bandwidth.

The Prefill (prompt processing) phase is compute-intensive and relies on the GPU's parallel matrix computation capability; the Decode (generation) phase is memory-bandwidth-intensive and depends more on VRAM bandwidth.

The prefill phase requires processing all input tokens at once and is a compute-intensive operation, whereas the decode phase generates only one token per step and is a memory-bandwidth-intensive operation.

The computational complexity of the prefill stage has a quadratic relationship with sequence length (O(n²)).

The Prefill stage of LLM inference is a compute-intensive operation whose primary bottleneck is GPU/NPU compute (FLOPS).

The Neural Accelerator's speedup for Prefill is approximately 3–5 times, far exceeding its speedup for Decode, which benefits more from increased memory bandwidth.

The Prefill stage of autoregressive model inference processes the entire prompt at once, which is a standard GEMM operation with high arithmetic intensity and is compute-bound.

The computational cost of Prefill scales approximately linearly to quadratically with input length, making it one of the main sources of inference latency.

The Prefill stage is the first step of large language model inference, in which the model performs a complete forward computation in parallel over all input tokens to generate the KV Cache for every layer.

Separating Prefill and Decode to run on different resources is the mainstream architecture for optimizing resource utilization in current large-scale inference services.

In the inference pipeline, the Prefill stage is compute-intensive and time-consuming, while the Decode stage performs token-by-token autoregressive generation and is sensitive to latency.