SGLang v0.5.18 Released: Performance Breakthroughs Across 710 PRs

SGLang v0.5.18 Update Overview

SGLang, the high-performance LLM inference engine, has landed a major version update with v0.5.18. This release is the culmination of 710 PRs submitted by 212 contributors, making it one of the most massive community collaboration efforts in recent history. As a star open-source project with over 34.1k stars on GitHub, SGLang has long been known for its extreme inference performance and rapid support for the latest models.

This update not only adds support for multiple multimodal and diffusion models, but also delivers substantial performance optimizations across core areas including startup acceleration, tensor parallelism, quantization, and speculative decoding. For teams deploying LLM inference services, v0.5.18 offers tangible efficiency improvements.

New Model Support: Comprehensive Expansion Across Multimodal and Diffusion Models

This version continues to expand the model ecosystem, covering three major categories: autoregressive, multimodal, and diffusion models. Newly supported models include:

• Autoregressive multimodal models: Muse Glimmer
• Autoregressive models: Intern-S2-Mobius
• Video and image diffusion models: SANA-Video, LingBot-Video-MoE, LTX-2.5, Cosmos3 Edge & Distilled, LongCat-Image

Beyond these, the team has also provided cookbook recipes for Qwen3.8 series, Ling-3.0, Nemotron 3.5 Lightning, Dots3-Note, and DeepSeek-V4-Pro-0813. This "adapt-on-release" cadence is exactly what keeps SGLang ahead in the inference engine race — it allows developers to deploy cutting-edge models into production environments immediately.

Startup Acceleration: Overlapped Checkpoint Staging Delivers 2.38x Speedup

One of the core highlights of this update is overlapped checkpoint staging. The traditional model loading workflow is sequential: first load weights from storage, then capture the CUDA graph. The new mechanism allows checkpoint pages to be staged from storage in parallel during CUDA graph capture.

The benchmark results are impressive:

• Qwen3-32B on H100 with prefetch mode achieves 8.6%-11.7% faster startup compared to the sequential approach
• Compared to the default plain loading method, this delivers a 2.38x speedup (35.6s vs 84.8s)

This feature can be enabled via --startup-weight-load-mode overlap. For inference clusters that require frequent scaling or cold starts, the practical value of this optimization is substantial.

Tensor Parallelism and Distributed Communication Optimizations

At the distributed inference layer, v0.5.18 introduces several key communication optimizations that directly reduce latency overhead in multi-GPU inference.

TP LMHead All-to-All Communication Merging

For pure DP dp-attention scenarios, the TP LMHead's original allgather + scatter two-step operation has been merged into a single all-to-all communication. In DeepSeek-V4-Pro B200 decode tests:

• LMHead latency dropped from 320μs to 169μs
• TPOT (time per output token) improved from 36.97ms to 35.67ms

FlashInfer MNNVL Pure AllReduce

Non-fused allreduce scenarios can now reuse the FlashInfer MNNVL workspace instead of falling back to NCCL. On the Blackwell architecture, DeepSeek-V4-Flash TP4 decode achieves up to +6.9% performance improvement at small batch sizes.

This feature is automatically enabled for DeepSeek-V3/V3.2/V4, and can be manually activated for other models via --enable-flashinfer-pure-allreduce.

Deep Optimization for AMD and Chinese-Developed Models

Notably, SGLang continues to push forward on heterogeneous hardware support, with significant optimization effort for the AMD platform and Chinese-developed large models in this release.

NVFP4 Checkpoint Quantization Conversion on AMD

With --quantization quark_mxfp4, the engine dequantizes ModelOpt and Quark NVFP4 weights at load time and requantizes them to MXFP4, all without ever holding a full-precision copy. On models including MiniMax-M2.7, GLM-5.1, Kimi-K2.6, Qwen3.5-397B, and DeepSeek-R1, GSM8K accuracy recovery reaches 97.5%-100.2%.

Deep Tuning of Kimi K3 for MI355X

For Kimi K3 inference on AMD MI355X, SGLang implemented multi-level optimizations:

• Replaced the previous MHA path (which repeatedly read shared latent variables per head) with a grouped-head MLA verification kernel, achieving 1.37-1.77x throughput improvement and 1.45-2.42x ITL improvement at concurrency levels of 2-32
• The AITER MLA prefill kernel was adapted for K3's 12-head structure, reducing TTFT by up to 14.9%
• gfx950-tuned decode-phase geometry delivers an additional 46-73% ITL improvement at 68k input
• GSM8K accuracy remains stable at 0.951-0.957 under these optimizations

Unified Cache Management and Developer Experience Improvements

This update unifies the compiled kernel cache directory. Caches for Triton, FlashInfer, Inductor, DeepGEMM, and the CUDA driver are all now consolidated under SGLANG_CACHE_DIR.

Note: This is a breaking change. The first startup after upgrading will trigger recompilation, but subsequent cache management will be much cleaner and more unified.

Additionally, the engine now freezes GC (garbage collection) after service warm-up, reducing runtime jitter. The scheduler has also been heavily optimized, including:

• O(1) slot allocation
• Batched cache eviction
• DP attention scheduler single D2H copy convergence

These improvements further reduce host-side overhead and improve service stability.

Dependency Upgrades and Companion Updates

The companion dependency stack has also been updated to the latest versions:

Dependency	Version
PyTorch	2.13.0 (with triton 3.7.1)
FlashInfer	0.6.17
CuTeDSL	4.6.2 (fixes FA4 launch regression on Blackwell)
DeepEP	Now installed from sgl-deep-ep wheels
sgl-kernel	0.4.6.post1

Conclusion: Is SGLang v0.5.18 Worth Upgrading To?

SGLang v0.5.18 is a substantial major release. From startup acceleration and communication optimization to cross-hardware quantization support, nearly every aspect reflects the engineering team's relentless pursuit of inference efficiency. The deep adaptation for AMD MI355X, Blackwell architecture, and frontier models like DeepSeek-V4 and Kimi K3 demonstrates SGLang's technical breadth and responsiveness in the inference engine landscape.

For production deployment teams, overlapped checkpoint loading and the suite of parallelism optimizations translate directly into lower latency and higher throughput. For model researchers, rapid new model support significantly shortens the path from paper to production.

Teams currently using SGLang are encouraged to evaluate upgrading, while keeping an eye on breaking changes such as the cache directory restructuring.