Prefill

The Prefill phase is a compute-intensive operation, while the Decode phase is a memory-bandwidth-intensive operation.

P/D Disaggregation (Prefill/Decode Disaggregation) deploys the Prefill and Decode stages on different nodes to independently scale according to their respective computational characteristics, while simultaneously optimizing throughput and time to first token (TTFT).

The Prefill phase is a compute-intensive task with high demands on compute power; the Decode phase is a memory-access-intensive task that is sensitive to GPU memory bandwidth.

Large model inference is divided into two phases, Prefill and Decode; Prefill is a major component of Time to First Token (TTFT), while Decode corresponds to Time Per Output Token (TPOT).

The prefill stage of Transformer inference is primarily bottlenecked by compute (FLOPS), while the decode stage is primarily bottlenecked by memory bandwidth.

TTFT is determined by the Prefill phase and is compute-bound, while TPOT corresponds to the Decode phase and is memory-bandwidth-bound.

The prefill stage of LLM inference processes all input tokens in parallel and generates the initial KV Cache; it is a compute-intensive operation and the direct determinant of TTFT.

传统混合部署中Prefill阶段的大批量计算会阻塞Decode阶段的token生成，导致TTFT波动剧烈

LLM 推理分为预填充阶段（Prefill，计算密集型）和解码阶段（Decode，内存带宽密集型）两个阶段

The inference framework uses a read-write separation architecture: the Prefill stage uses ordinary 8-card machines and the Decode stage uses large machines with 64 or 128 cards; the minimum configuration is 64 read cards and 64 write cards, 128 cards in total, and reads and writes can be deployed in different data centers connected by fiber optics.

The prefill phase performs one forward computation over the entire input to produce KV caches for all positions, and is compute-bound; the decode phase generates tokens autoregressively one by one and is memory-bandwidth-bound.

Transformer inference is divided into two stages, prefill and decode: prefill processes the entire input prompt in parallel at once and is mainly compute-bound; decode generates output tokens one by one and is mainly memory-bandwidth-bound.

The prefill phase feeds the entire input prompt into the model at once, attention computation can be fully parallelized, and it is a compute-intensive workload where GPU utilization is near saturation but latency scales approximately linearly with input length.

The prefill phase processes the complete input context, computes attention for all input tokens, and generates the KV cache; it is a compute-intensive operation whose time consumption is strongly correlated with input length.

The prefill stage refers to the process in which the model processes the input prompt all at once; the computation is proportional to input length, and in long-document scenarios the time to first token (TTFT) is significantly prolonged.

Disaggregated inference decouples the compute-intensive prefill stage from the memory-bandwidth-intensive decode stage, allocating them to separate hardware resource pools best suited to each for independent scaling.

The prefill stage is highly parallel with large throughput but latency that grows with context length, while the decode stage generates tokens auto-regressively and its speed is limited by memory bandwidth.

The Prefill (prompt processing) phase is compute-intensive and relies on the GPU's parallel matrix computation capability; the Decode (generation) phase is memory-bandwidth-intensive and depends more on VRAM bandwidth.

The prefill phase requires processing all input tokens at once and is a compute-intensive operation, whereas the decode phase generates only one token per step and is a memory-bandwidth-intensive operation.

The computational complexity of the prefill stage has a quadratic relationship with sequence length (O(n²)).