Unverified50% confidenceFactExact time
CUDA graphs record the kernel launch sequence as a graph and submit it in one go, reducing CPU launch overhead, which is especially beneficial for small-batch low-latency scenarios.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedCUDA Graph is a performance optimization technology introduced by NVIDIA in CUDA 10 that reduces CPU-GPU synchronization latency through a record-and-replay mechanism.77% similarUnverifiedThe core idea of the dlmrt inference engine is to capture the entire denoising loop into a single CUDA graph and execute it directly on the GPU, avoiding launching and synchronizing from the CPU at every step.77% similarUnverifiedBecause the prefill phase must handle variable-length inputs, it often needs to capture CUDA graphs separately for multiple sequence lengths, further amplifying the recording time.72% similarUnverifiedAt startup, vLLM typically performs CUDA graph capture only for the decode phase, and has a more aggressive trimming strategy for the set of captured batch sizes, so its cold-start graph capture phase is often shorter than SGLang's.72% similarUnverifiedcuda-checkpoint能将GPU上运行进程的完整状态（包括显存数据、CUDA上下文、设备句柄）序列化保存至磁盘并快速还原至GPU继续运行71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1061239API
curl https://kongchang.com/api/v1/knowledge/claims/1061239MCP
get_claim(id=1061239)