Unverified50% confidenceFactExact time
Because the prefill phase must handle variable-length inputs, it often needs to capture CUDA graphs separately for multiple sequence lengths, further amplifying the recording time.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedOverlapped checkpoint staged loading allows checkpoint pages to be loaded in parallel from storage in stages during CUDA graph capture.76% similarUnverifiedThe cost of CUDA graphs is that the recording phase itself requires running the entire graph through once or even multiple times, and it has limited support for dynamic shapes and dynamic control flow.73% similarUnverifiedInference framework runtime overhead (CUDA context initialization, operator fusion caching, memory fragmentation, etc.) typically consumes an additional 0.5–2 GB70% similarUnverifiedCUDA Graph is a performance optimization technology introduced by NVIDIA in CUDA 10 that reduces CPU-GPU synchronization latency through a record-and-replay mechanism.69% similarUnverifiedAt startup, vLLM typically performs CUDA graph capture only for the decode phase, and has a more aggressive trimming strategy for the set of captured batch sizes, so its cold-start graph capture phase is often shorter than SGLang's.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1012434API
curl https://kongchang.com/api/v1/knowledge/claims/1012434MCP
get_claim(id=1012434)