Prefill
One of the two main phases of large language model inference, responsible for simultaneously computing attention across all tokens in the input prompt. Computational cost scales quadratically with sequence length, making this a compute-intensive operation.
Core Facts
Time to First Token (TTFT) refers to the time interval between a user sending a request and the model outputting the first token, and is primarily constrained by the duration of the prefill stage.
The KV Cache mechanism can cache repeated prefix content, and the more concise the system prompt, the higher the cache hit rate.
LLM 推理分为预填充阶段（Prefill，计算密集型）和解码阶段（Decode，内存带宽密集型）两个阶段
Timeline (last 90 days)
A machine without a dedicated GPU cannot ingest large prompts in a reasonable time, so a second host with a GPU is needed to solve the prefill problem.
The prefill stage is a compute-intensive operation primarily limited by GPU compute, while the decode stage is a memory-bandwidth-intensive operation primarily limited by VRAM bandwidth.
LLM inference requests are divided into two phases—Prefill (input) and Decode (output); Prefill can read inputs in parallel with extremely high throughput, while Decode requires token-by-token serial inference and occupies significantly more GPU time.
Time to First Token (TTFT) refers to the time interval between a user sending a request and the model outputting the first token, and is primarily constrained by the duration of the prefill stage.
The KV Cache mechanism can cache repeated prefix content, and the more concise the system prompt, the higher the cache hit rate.
All Facts (6)
Time to First Token (TTFT) refers to the time interval between a user sending a request and the model outputting the first token, and is primarily constrained by the duration of the prefill stage.
90%VerifiedThe KV Cache mechanism can cache repeated prefix content, and the more concise the system prompt, the higher the cache hit rate.
80%Partially VerifiedLLM 推理分为预填充阶段（Prefill，计算密集型）和解码阶段（Decode，内存带宽密集型）两个阶段
65%UnverifiedA machine without a dedicated GPU cannot ingest large prompts in a reasonable time, so a second host with a GPU is needed to solve the prefill problem.
50%UnverifiedThe prefill stage is a compute-intensive operation primarily limited by GPU compute, while the decode stage is a memory-bandwidth-intensive operation primarily limited by VRAM bandwidth.
50%UnverifiedLLM inference requests are divided into two phases—Prefill (input) and Decode (output); Prefill can read inputs in parallel with extremely high throughput, while Decode requires token-by-token serial inference and occupies significantly more GPU time.
50%