vLLM v0.29.0 Released: Model Runner V2 Becomes the Default Engine, Inference Performance Gets a Major Boost

vLLM v0.29.0 Core Changes: Model Runner V2 Goes Default

The open-source large model inference framework vLLM has released version 0.29.0, and the most notable change is that Model Runner V2 is now the default engine for all models. This decision signals that after validation across multiple releases, the V2 architecture is stable enough to replace the original runtime and handle all inference tasks.

This release brings together 594 commits from 277 contributors, 91 of whom are first-time participants. This level of community activity is rare in the open-source inference framework space, and it reflects how widely vLLM has been adopted in production environments.

For teams operating large-scale inference services, a change to the default engine is worth paying close attention to. It typically comes with adjustments to scheduling logic, memory management, and batching strategies — all of which can directly affect throughput and latency.

Model-Specific Optimizations: Deep Adaptation for Kimi and DeepSeek

This update includes targeted optimizations for several widely used models. On Kimi-K3, the team carried out another round of performance tuning, further compressing the latency tail down to the Mamba metadata level — the kind of low-level optimization that can deliver meaningful efficiency gains in long-context scenarios.

For DeepSeek-V4, shared experts have been fused into the MegaMoE structure. Fusing shared experts in Mixture-of-Experts (MoE) models is an important direction in inference acceleration. By reducing redundant computation and memory transfers, it lowers inference costs while preserving model capability.

These aren't vague "performance improvements" — they're engineering work targeted at specific model architectures. It reflects a broader trend of vLLM evolving from a general-purpose inference framework into one that deeply adapts to leading models, which is a direct win for developers working with these models.

Three Key Technical Upgrades

Mamba Prefix Caching

Prefix caching for the Mamba architecture can now preserve internal prefill checkpoints, resulting in a measured 9%–25% improvement in time-to-first-token (TTFT). TTFT is a core metric for interactive application experience. For chatbots, real-time Q&A, and similar use cases, gains in this range are noticeable to end users.

Improved Observability for Speculative Decoding

Speculative Decoding can now report per-request acceptance stats via the API. The actual speedup from speculative decoding is highly dependent on how well the draft model matches the target model, and acceptance rate is the key metric for evaluating its effectiveness. Exposing this metric at the API level gives developers the ability to monitor and fine-tune at a granular level — a practical improvement in observability.

P2P Backend for RL Weight Synchronization

RL weight synchronization now supports a sharded_rdt P2P backend. With this mechanism, each worker only pulls the TP/EP (tensor parallel / expert parallel) shards that belong to it, rather than syncing the full weights. This is particularly valuable for training-inference co-located pipelines like RLHF, where weights are updated frequently, and it significantly reduces communication overhead in distributed environments.

New Model Support

v0.29.0 adds out-of-the-box support for several new models, including Hy4-preview, Qwen3.8-Flash-Next, GraniteSWA, and NemotronH Omni Reasoning V3. Continuously expanding model coverage is fundamental to vLLM maintaining its competitive edge — the value of an inference framework depends largely on its ability to support the community's newest and most relevant models as quickly as possible.

This list spans from lightweight variants in the Qwen series to models with Omni multimodal reasoning capabilities, demonstrating that vLLM's architecture handles a diverse range of model types with solid generality.

What This Means for Developers

Overall, v0.29.0 is an iteration focused on performance and stability. The promotion of Model Runner V2 to default marks a maturation of the underlying architecture. The targeted optimizations for Kimi and DeepSeek, combined with improvements to Mamba prefix caching, speculative decoding observability, and P2P RL weight synchronization, collectively advance inference efficiency, monitoring capability, and training integration.

For teams running vLLM in production, it's worth evaluating any behavioral changes that may come with the default engine switch before upgrading, and validating throughput and latency metrics in a test environment first. For developers focused on deploying new models, the expanded model support list provides more ready-to-use options out of the box.