[KongchangAI]
Concept分块预填充

Chunked Prefill

An inference optimization technique that splits the prefill stage of long inputs into multiple chunks processed progressively, reducing peak memory usage and improving throughput.

Timeline (last 90 days)

Oct 19

Chunked prefill limits each stall to the chunk size by splitting the prompt into chunks and interleaving them with decode steps, but it cannot eliminate the stalls entirely.

Unverified50%
Oct 19

vLLM's documentation explicitly states that P/D disaggregation reduces ITL tail latency more reliably than chunked prefill.

Unverified50%

All Facts (2)

Source Articles