Prefill/Decode Disaggregation
An architectural innovation in large language model inference that deploys the prefill and decode computation stages on separate nodes. Each stage can be scaled independently according to its distinct computational characteristics, enabling simultaneous optimization of throughput and latency in production inference systems.
Timeline (last 90 days)
vLLM's official documentation states that disaggregated prefill (P/D disaggregation) allows independent tuning of time to first token (TTFT) and inter-token latency (ITL), and also gives more reliable control over ITL tail jitter, but disaggregated prefill does not improve throughput.
vLLM's documentation explicitly states that P/D disaggregation reduces ITL tail latency more reliably than chunked prefill.
Under the assumptions of a 70B dense model, 4 GPUs sharing prefill, 400 TFLOPS of effective compute per GPU, the link running at line rate and no protocol overhead counted, the ratio of transfer time to prefill time depends only on model shape, link bandwidth and prefill throughput, and is independent of prompt length.
The real comparison for P/D disaggregation is not transfer cost versus zero cost, but transfer cost versus the harm that same-machine prefill does to p99 ITL under real request patterns.
All Facts (4)
vLLM's official documentation states that disaggregated prefill (P/D disaggregation) allows independent tuning of time to first token (TTFT) and inter-token latency (ITL), and also gives more reliable control over ITL tail jitter, but disaggregated prefill does not improve throughput.
50%UnverifiedvLLM's documentation explicitly states that P/D disaggregation reduces ITL tail latency more reliably than chunked prefill.
50%UnverifiedUnder the assumptions of a 70B dense model, 4 GPUs sharing prefill, 400 TFLOPS of effective compute per GPU, the link running at line rate and no protocol overhead counted, the ratio of transfer time to prefill time depends only on model shape, link bandwidth and prefill throughput, and is independent of prompt length.
50%UnverifiedThe real comparison for P/D disaggregation is not transfer cost versus zero cost, but transfer cost versus the harm that same-machine prefill does to p99 ITL under real request patterns.
50%