CUDA graphs
A technology in NVIDIA CUDA that records a sequence of kernel launches as a graph and submits it at once, reducing CPU launch overhead and improving performance in small-batch, low-latency scenarios.
Timeline (last 90 days)
CUDA graphs record the kernel launch sequence as a graph and submit it in one go, reducing CPU launch overhead, which is especially beneficial for small-batch low-latency scenarios.
The Laya decision model achieves up to a 4.1× speedup for short requests, with the speed gains primarily coming from the introduction of a marker-only head and CUDA graphs
All Facts (2)
CUDA graphs record the kernel launch sequence as a graph and submit it in one go, reducing CPU launch overhead, which is especially beneficial for small-batch low-latency scenarios.
50%UnverifiedThe Laya decision model achieves up to a 4.1× speedup for short requests, with the speed gains primarily coming from the introduction of a marker-only head and CUDA graphs
50%