Chunked Prefill
An inference optimization technique that splits the prefill stage of long inputs into multiple chunks processed progressively, reducing peak memory usage and improving throughput.
Timeline (last 90 days)
Chunked prefill limits each stall to the chunk size by splitting the prompt into chunks and interleaving them with decode steps, but it cannot eliminate the stalls entirely.
vLLM's documentation explicitly states that P/D disaggregation reduces ITL tail latency more reliably than chunked prefill.
All Facts (2)
Chunked prefill limits each stall to the chunk size by splitting the prompt into chunks and interleaving them with decode steps, but it cannot eliminate the stalls entirely.
50%UnverifiedvLLM's documentation explicitly states that P/D disaggregation reduces ITL tail latency more reliably than chunked prefill.
50%