[KongchangAI]
ConceptPrefill/Decode Disaggregation / 预填充解码分离

Prefill/Decode Disaggregation

An architectural innovation in large language model inference that deploys the prefill and decode computation stages on separate nodes. Each stage can be scaled independently according to its distinct computational characteristics, enabling simultaneous optimization of throughput and latency in production inference systems.

Timeline (last 90 days)

Oct 19

vLLM's official documentation states that disaggregated prefill (P/D disaggregation) allows independent tuning of time to first token (TTFT) and inter-token latency (ITL), and also gives more reliable control over ITL tail jitter, but disaggregated prefill does not improve throughput.

Unverified50%
Oct 19

vLLM's documentation explicitly states that P/D disaggregation reduces ITL tail latency more reliably than chunked prefill.

Unverified50%
Oct 19

Under the assumptions of a 70B dense model, 4 GPUs sharing prefill, 400 TFLOPS of effective compute per GPU, the link running at line rate and no protocol overhead counted, the ratio of transfer time to prefill time depends only on model shape, link bandwidth and prefill throughput, and is independent of prompt length.

Unverified50%
Oct 19

The real comparison for P/D disaggregation is not transfer cost versus zero cost, but transfer cost versus the harm that same-machine prefill does to p99 ITL under real request patterns.

Unverified50%

All Facts (4)

Source Articles