vLLM v0.27.0 Deep Dive: Full Kimi K3 Support and DeepSeek-V4 Performance Optimizations

vLLM is one of the most widely adopted large model inference engines today, and every major release sends ripples across the entire AI infrastructure ecosystem. The vLLM team has officially released v0.27.0. According to the official release notes, this update incorporates 561 commits from 242 contributors, including 64 first-time contributors — a testament to the community's vitality and a signal that this is a particularly substantial iteration.

This article takes a deep dive into the key highlights of this release across model support, engine core, hardware performance, and large-scale serving.

Full-Stack Kimi K3 Support: A Complete Single-Version Landing

The most headline-grabbing feature of this release is complete, full-stack support for the Kimi K3 model — with all related components landing in a single version. This is uncommon in inference engine model integration, which typically requires multiple versions to progressively polish. The Kimi K3 support covers the entire stack from bottom to top:

• Core layer: Model files, kernel implementations, and dedicated AttnRes kernels;
• Frontend layer: Both Python and Rust frontend support;
• Quantization & optimization: DeepGEMM integration, compressed-tensors quantization checkpoints, and DSpark AR fusion;
• Architecture optimization: An option to shard (rather than replicate) the shared expert, reducing GPU memory overhead.

This "all-in-one" approach means users can deploy Kimi K3 seamlessly after upgrading without waiting for follow-up patches, demonstrating the maturity of vLLM's model integration engineering.

New Model Support at a Glance

Beyond Kimi K3, this release adds support for several other models across text, multimodal, and embedding use cases:

• Qwen3.5 series, including dense and MoE variants for pure text, with EVS video token pruning support;
• K-EXAONE-2.0-750B-A37B, a large-scale MoE model;
• VaultGemma, supported via the Transformers modeling backend;
• jina-embeddings-v5-text-nano, an embedding model using an EuroBERT encoder backbone.

Engine Core Upgrades: Eliminating First-Request Compilation Stalls

For production deployments, the "cold start" experience of an inference engine is critical. Previously, vLLM would often exhibit noticeable latency spikes on the very first request due to just-in-time (JIT) compilation.

v0.27.0 addresses this pain point with a new JIT warmup infrastructure that warms up all Triton kernels owned by the runner before the first request arrives, eliminating compilation stalls on initial requests. Renderer warmup has also been improved. These changes will significantly enhance the responsiveness of a service at startup.

Model Runner V2 Expanded to Non-Generative Workloads

A major architectural evolution is the significant expansion of Model Runner V2 (MRV2) beyond traditional text generation tasks to cover a much broader range of workloads:

• Encoder-only attention support;
• Sequence pooling for embedding and classification;
• Encoder token classification and token embedding;
• BGE-M3 pooling;
• Multimodal execution on CPU;
• Multi-layer MTP speculator.

This signals that vLLM is evolving from a pure generative inference engine into a unified inference platform capable of handling generation, embedding, classification, and more.

Comprehensive KV Cache Offloading Overhaul

On the KV offloading front, this release builds a more complete tiered caching system, including a generic P2P secondary storage layer, per-request tiered filtering (TierFilter/TierMatcher), a pluggable eviction policy factory (CachePolicyFactory), and MLA KV deduplication with single-copy layout optimization. These improvements have real-world implications for memory and GPU memory efficiency in long-context, high-concurrency scenarios.

JIT (Just-In-Time) compilation refers to compiling code into machine instructions at runtime rather than ahead of time. vLLM relies heavily on Triton — a domain-specific language for writing GPU kernels — and its kernels are only compiled on first invocation, a process that can take tens of seconds. This is especially problematic for online serving: the first real user request often experiences an abnormally long response time, sometimes long enough to trigger client-side timeouts. The warmup mechanism works by proactively triggering the compilation and caching of all kernels using dummy inputs during the server startup phase, before any real traffic is accepted — so that subsequent real requests hit already-compiled kernels and avoid latency spikes.

KV (Key-Value) cache is a core data structure in Transformer attention, storing the key and value vectors generated at each layer for each token to avoid redundant computation during autoregressive decoding. KV offloading refers to migrating KV caches that are temporarily inactive from GPU memory to CPU memory or even NVMe storage, enabling longer contexts or higher concurrency. Tiered caching systematizes this idea further: through mechanisms like a P2P secondary storage layer and pluggable eviction policies, it dynamically schedules cache data across GPU memory, CPU memory, and external storage, balancing access speed against capacity. MLA (Multi-head Latent Attention) is a low-rank KV compression attention variant used in the DeepSeek model series; its KV structure differs from standard MHA and therefore requires dedicated deduplication and layout optimizations.

Deep Performance Optimization for DeepSeek-V4 and Hardware Adaptation

Performance optimization is always a centerpiece of vLLM releases, and one major focus this time is a series of performance breakthroughs for DeepSeek-V4. The official release notes list numerous specific data points:

• Sequence parallelism support;
• ~2x kernel improvement by skipping empty c128 launches;
• 3.4% end-to-end TTFT (Time to First Token) improvement by skipping unnecessary topk/router during the decode phase;
• 3.9% end-to-end TTFT improvement through workspace reuse;
• 1.88x kernel improvement by removing redundant full kernels;
• 1.0% end-to-end improvement from adaptive topk width;
• 448 MiB GPU memory savings in PP buffers.

While these might seem like incremental micro-optimizations, they accumulate into meaningful cost and latency gains for teams running DeepSeek-V4 at scale.

FlashAttention 4 Integration and Next-Generation Hardware Readiness

FlashAttention 4 integration on SM100 is deepened with the addition of FP8 KV cache support and headdim-256 support. vLLM is also actively laying the groundwork for next-generation hardware:

• Enabling the sm_107 target for the NVIDIA Rubin architecture, with an NVLink all-reduce path on SM107;
• Enabling ROCm gfx1250, continuing to strengthen support for the AMD ecosystem.

On the CPU side, there are also notable additions, including INT8 fused MoE kernels for Arm CPUs and oneDNN INT8 GEMM optimizations for s390x, further broadening vLLM's deployment reach.

FlashAttention is an IO-aware algorithm that rewrites the attention computation using tiling techniques to reduce HBM memory read/write operations. It dramatically reduces the memory bandwidth bottleneck of attention while remaining mathematically equivalent. FlashAttention 4 is its latest iteration, deeply optimized for the Tensor Cores and asynchronous pipeline features of NVIDIA Hopper (SM90) and later architectures, with FP8 precision support for even higher throughput. NVIDIA Rubin is the next-generation GPU microarchitecture after Hopper, identified by the SM107 architecture designation; SM100 corresponds to the Blackwell architecture. Enabling compilation targets for new architectures (sm_107) ahead of time means vLLM users will get native performance on Rubin GPUs immediately upon availability, without waiting for additional engine-level adaptation work.

Large-Scale Serving and the Continued Evolution of the Rust Frontend

For enterprise-scale deployments, v0.27.0 introduces a (simplified) fault tolerance framework designed specifically for DP+EP external load balancer deployment scenarios, along with async readiness support for elastic EP scaling. This marks an important step forward for vLLM in terms of service resilience.

On the disaggregation front, this release adds NIXL P/D support for hybrid models (e.g., MLA+SSM), along with heterogeneous P/D block sizes and MoRIIO heterogeneous TP↔DP prefill/decode read routing.

The Rust frontend continues to grow significantly. This release adds a complete gRPC control plane, including engine-aware health reporting, abort control, service and model discovery, and KV event source discovery. Additionally, vllm-bench has been integrated into the vllm CLI, with users able to optionally delegate vllm bench serve execution to the Rust implementation. Taken together, these moves clearly signal that vLLM is rebuilding its performance-sensitive serving layer components in Rust.

DP+EP is a parallelism strategy combination for large-scale MoE model deployment: DP (Data Parallelism) distributes different requests across different model replicas, while EP (Expert Parallelism) distributes different experts in the MoE layer across different GPUs. Together, they enable efficient utilization of compute across very large clusters. P/D Disaggregation (Prefill/Decode Disaggregation) is a deployment architecture that assigns the prefill phase (processing the input prompt) and the token-by-token decode phase to different hardware nodes, enabling independent optimization for the two phases' very different computational characteristics — improving both throughput and latency simultaneously. NIXL is a cross-node tensor transfer library from NVIDIA, designed specifically for efficiently transferring KV caches in P/D disaggregation scenarios.

Dependency Upgrade Notice: PyTorch 2.13 Breaking Changes

Finally, a critical heads-up for developers: this release upgrades to PyTorch 2.13.0, paired with torchvision 0.28.0 and Triton 3.7.1. The official notes explicitly flag this as a breaking environment change. XPU and CPU platforms are also updated to torch 2.13.

Additionally, Transformers is upgraded to 5.14.1, FlashInfer is updated to 0.6.16.post3, and NCCL is upgraded to 2.30.7 (enabling DeepEPv2 in official images). Teams planning to upgrade should carefully assess the impact of these dependency changes on their existing environments and run thorough compatibility tests.

Summary

vLLM v0.27.0 is an exceptionally content-rich release. From the full-stack, single-version landing of Kimi K3, to Model Runner V2's expansion into non-generative tasks; from the deep performance tuning of DeepSeek-V4, to forward-looking support for next-generation hardware like Rubin; and the continued evolution of the Rust frontend — these directions collectively paint a clear picture of vLLM's trajectory: evolving from a "high-performance inference engine" into a "universal, resilient, cross-hardware inference infrastructure." For teams building AI services on vLLM, this is a release that warrants serious evaluation and upgrade planning.