vLLM v0.28.0 Released: 584 Commits Deliver Full-Stack Inference Optimization

vLLM v0.28.0 Major Release

As one of the most widely adopted inference and serving engines for large language models, vLLM has officially released version v0.28.0. According to data published on Twitter, this update merged 584 commits from 270 contributors, including 76 first-time contributors. These numbers reflect not only the vitality of the vLLM project, but also the open-source community's sustained investment in LLM inference infrastructure.

For developers and organizations deploying large models with vLLM, the core value of this release lies in its "full-stack optimization" philosophy — rather than patching isolated features, it systematically refines performance across multiple layers to accommodate the latest generation of model architectures.

Deep Optimizations for Next-Generation Models

Full-Stack Optimization and Sparse MLA Support

One major highlight of this release is full-stack optimization targeting Kimi-K3. Full-stack optimization means coordinated improvements across operator scheduling, memory management, and the inference pipeline — a level of integration that typically delivers far greater end-to-end gains than isolated improvements.

Notably, DeepSeek^Ad-V4's sparse MLA (Multi-head Latent Attention) is now end-to-end available, covering three key scenarios:

• Plain decode: Standard autoregressive generation;
• MTP (Multi-Token Prediction): Predicts multiple tokens at once to increase generation throughput;
• DSpark: An acceleration mechanism within the DeepSeek ecosystem.

Sparse attention mechanisms have become a critical approach to reducing the cost of long-context inference in recent years. MLA significantly lowers GPU memory usage by compressing the KV cache, and vLLM's complete implementation means developers can deploy these models in production at lower cost.

Speculative Decoding and Runtime Evolution

Smarter Speculative Decoding

Speculative Decoding is a key technique for accelerating LLM inference. The core idea is to use a smaller draft model to generate multiple candidate tokens, which the main model then verifies in batch — effectively amortizing the cost of generating each token.

v0.28.0 introduces DFlash2 and DSpark confidence-scheduled verification in this area. The latter is particularly noteworthy: traditional speculative decoding applies a uniform verification strategy to all draft tokens, while confidence-scheduled verification dynamically adjusts verification intensity based on how reliable each candidate token is. In theory, this reduces unnecessary computation while maintaining output quality, thereby improving acceptance rates.

Model Runner V2 Capability Expansion

The next-generation Model Runner V2 gains two important capabilities in this release:

• E/P/D disaggregation: Decoupled deployment of the Encode/Prefill/Decode stages. Separating Prefill and Decode onto different resources is the mainstream architecture for optimizing resource utilization in large-scale inference services, effectively alleviating waste caused by the differing computational characteristics of each stage.
• Weight offloading: Allows portions of model weights to be offloaded beyond GPU memory, making it possible to run larger models under constrained VRAM conditions.

Tiered KV Caching and Expanded Model Ecosystem

Disk Tier Added to KV Offloading

KV cache management has always been a core strength of vLLM (its PagedAttention technology was built around this). In this update, Tiered KV offloading gains a disk tier and supports out-of-tree secondary cache layers.

This means the KV cache is no longer limited to GPU memory and host RAM — it can now spill further down to disk storage. For scenarios with massive KV cache footprints such as ultra-long contexts and multi-turn conversations, this tiered design offers more flexible tradeoffs between cost and performance. The open out-of-tree extension mechanism also makes it easy for enterprises to integrate custom storage backends.

New Model Support

vLLM has consistently been praised for its rapid adoption of new models. Models newly supported in this release include:

• Muse Glimmer
• Ling 3.0 Flash
• Dots3 NOTE
• Interns2mobius

The continued expansion of model coverage allows vLLM to serve as a unified inference backbone for a diverse model ecosystem, reducing the migration cost for developers switching between different models.

Summary and Outlook

Taken together, vLLM v0.28.0 is not a simple feature pile-up, but a systematic upgrade centered on inference efficiency. From end-to-end sparse MLA support and confidence-scheduled speculative decoding, to E/P/D disaggregated architecture and disk-level KV offloading — each improvement directly targets real production pain points: GPU memory bottlenecks, throughput ceilings, and deployment costs.

For teams currently deploying or planning to deploy large models in production, the disaggregated deployment and tiered caching capabilities in this release are especially worth evaluating. With over 270 contributors continuing to push the project forward, vLLM is cementing its status as the de facto standard for LLM inference. Interested readers are encouraged to review the official release notes and benchmark the relevant new features against their own models and hardware configurations.