vLLM v0.28.0rc2 Released: A Deep Dive into DFlash2 Speculative Decoding
vLLM v0.28.0rc2 introduces DFlash2 speculative decoding with local convolution and candidate selector for faster LLM inference.
vLLM has released release candidate v0.28.0rc2, with the headline feature being the new DFlash2 speculative decoding mechanism. Speculative decoding uses a lightweight draft mechanism to pre-generate multiple candidate tokens, which the target model verifies in parallel in a single forward pass — drastically reducing target model calls while preserving output distribution. DFlash2 consists of two modules: local convolution for low-cost capture of token dependency patterns, and a candidate selector that intelligently filters candidates to balance coverage against verification cost and improve acceptance rates. As an RC release, production deployments should await the stable version, but developers in latency-sensitive scenarios like chat and code generation can evaluate its benefits in test environments now.
vLLM Continues to Lead LLM Inference Acceleration
vLLM remains one of the most popular inference engines for large language models, with its GitHub repository accumulating over 90.9k stars and 21.7k forks — establishing it as the de facto high-performance inference standard in the industry. Recently, vLLM released the v0.28.0rc2 release candidate, with the most notable change being the introduction of DFlash2, a new speculative decoding implementation that combines two key technical components: local convolution and a candidate selector (PR #52816).
This article unpacks the core principles behind speculative decoding and explores the inference acceleration value that DFlash2 may deliver.
Speculative Decoding: A Critical Path to Faster LLM Inference
Why LLM Inference Needs Speculative Decoding
Autoregressive generation in large language models is inherently sequential — every token requires a full forward pass. This serial nature means inference speed is bottlenecked by memory bandwidth rather than compute, leaving GPU computational capacity significantly underutilized.
Speculative decoding was designed precisely to address this bottleneck. The core idea is to use a lightweight "draft model" or lightweight mechanism to rapidly generate multiple candidate tokens, and then have the main target model verify all of these candidates in parallel within a single forward pass. Accepted tokens are kept as-is; rejected tokens fall back to resampling. This approach dramatically reduces the number of target model calls while guaranteeing that the output distribution remains identical to the original model.
DFlash2's Technical Positioning and Core Modules
DFlash2, introduced in this update, is a new implementation within the speculative decoding framework. Based on the PR description, it consists of two key modules:
- Local convolution: This mechanism is likely used to capture local dependency patterns in token sequences, using low-cost convolution operations to quickly predict candidate tokens — effectively replacing or supplementing the role of a traditional draft model.
- Candidate selector: Responsible for filtering among multiple generated candidates, determining which ones proceed to the target model's parallel verification stage. This balances "coverage" against "verification cost."
This combination of lightweight prediction and intelligent filtering represents a further refinement of speculative decoding at the engineering implementation level.
Release Details and Engineering Best Practices
What a Release Candidate Means
The rc in v0.28.0rc2 stands for Release Candidate. These versions typically indicate that core features have completed development and entered a final stability validation phase — they have not yet been published as a stable release. Developers eager to try DFlash2 early can experiment in test environments, but production deployments should wait for the official stable release.
The release notes show that this update was merged into the release branch via cherry-pick (cherry-picked from commit b389ac2...) from the main branch, signed off by maintainer khluu. This reflects vLLM's disciplined branch management and release process — new features are developed and validated on mainline first, then selectively backported to release branches, ensuring the controllability of released versions.
Impact of DFlash2 on Real-World Deployments
For teams relying on vLLM for model serving, the most immediate benefit of speculative decoding features is reduced inference latency and improved throughput. This is especially true in latency-sensitive scenarios such as conversational applications and code generation, where speculative decoding can deliver several-fold improvements in token generation rate without sacrificing output quality.
It's worth noting that the speedup from speculative decoding is highly dependent on the "acceptance rate" of the draft mechanism. If candidate tokens are frequently rejected by the target model, the additional verification overhead can actually hurt performance. The design intent behind DFlash2's candidate selector is precisely to improve effective acceptance rates and reduce wasted verification cycles.
The Competitive Landscape of Inference Engines and the Road Ahead
Speculative decoding has become a central battleground for optimization across major inference engines. From Medusa and EAGLE to Lookahead Decoding, the industry continues to produce new speculative decoding variants. By integrating DFlash2 into its Spec Decode ecosystem, vLLM further expands the acceleration options available to users.
As large model applications scale up in production, inference cost has become a central concern for enterprises. Whoever can deliver lower latency and higher throughput while maintaining output quality will have a meaningful edge in deployment economics. vLLM's sustained investment in speculative decoding is a direct response to this trend.
For developers focused on LLM engineering, it's worth keeping a close eye on vLLM's release notes and running A/B tests under real business workloads to validate the actual gains from new features like DFlash2 in your specific use case.
Conclusion
Although vLLM v0.28.0rc2 is a release candidate, the DFlash2 speculative decoding mechanism it introduces is worth the industry's attention. The combination of local convolution and a candidate selector represents another valuable exploration in the engineering implementation of speculative decoding. On the track of inference acceleration, vLLM continues to maintain both the vitality of the open-source community and its technical leadership.
Related articles
DeepSeek V4 Pro Burning Through Credits Too Fast? The Hidden Logic Behind AI Model Pricing
Why does DeepSeek V4 Pro drain credits so fast while Flash barely moves? A deep dive into AI token billing, Pro vs. Flash pricing differences, and cost optimization tips.
RealPDE Competition Breakdown: The Frontier Challenge of AI-Powered Real-World Fluid Dynamics PDE Solving
A deep dive into the NeurIPS 2026 RealPDE Competition, covering the Sim2Real and LTTTA tracks, and how neural operators tackle real-world PIV and CFD fluid PDE challenges.
Building a Production-Grade 3DGS Training Library from Scratch: A Deep Dive into Full-GPU Residency and the Vulkan Stack
A veteran graphics engineer builds a production-grade 3DGS training library from scratch using C++23, CUDA, and Vulkan, achieving 60fps with 5M splats. Deep dive into its architecture and design.