Unverified50% confidenceFactExact time
The inference process is divided into the prefill stage and the decode stage: the prefill stage processes the entire prompt and saves the full KV Cache, and the decode stage generates subsequent tokens one by one and reuses the cache.
1
Sources
50%
Confidence
Long-term
Relevance
10/31/2026
First Seen
Sources
LLM Inference Bottlenecks and KV Cache: Core Mechanisms and Mainstream Optimization Approaches
bilibili先进编译实验室10/20/2026
Related Entities
Related Claims
UnverifiedIn the bidirectional KV transfer mechanism, prefill passes the initial KV to decode, and the incremental KV produced during decode generation also flows back to the prefill side, so that each round performs incremental prefill on top of the previous round rather than full recomputation.79% similarVerifiedLarge model inference is divided into two phases, Prefill and Decode; Prefill is a major component of Time to First Token (TTFT), while Decode corresponds to Time Per Output Token (TPOT).78% similarVerifiedLLM inference is divided into two phases: the prefill phase (compute-intensive) and the decode phase (memory-bandwidth-intensive).74% similarUnverifiedThe decode phase of large language model inference generates only one token at a time and is a memory-bandwidth-bound operation, because generating each token requires loading the entire model weights from memory once.71% similarVerifiedThe prefill stage of LLM inference processes all input tokens in parallel and generates the initial KV Cache; it is a compute-intensive operation and the direct determinant of TTFT.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1121453API
curl https://kongchang.com/api/v1/knowledge/claims/1121453MCP
get_claim(id=1121453)