Concept分块预填充

Chunked Prefill

An inference optimization technique that splits a long prompt into multiple chunks and interleaves them with decoding steps, thereby limiting the duration of inter-token latency spikes caused by prefill steps.

Timeline (last 90 days)

Oct 16

PagedAttention, Chunked Prefill, and prefix cache reuse can mitigate TTFT jitter, but cannot replace limiting context growth at the architectural level.

Unverified50%

All Facts (1)

Unverified

PagedAttention, Chunked Prefill, and prefix cache reuse can mitigate TTFT jitter, but cannot replace limiting context growth at the architectural level.

50%

Source Articles

One Machine Becomes Two: LLM Inference Disaggregation and the Engineering Logic Behind a 75% Throughput GainOct 19 vLLM P/D Disaggregation: The Calculation Rule Where KV Transfer Cost Determines BenefitOct 19