D3D: Decoupled Diffusion Drafting Achieves 5.71x Speedup with Asynchronous Speculative Decoding

Speculative decoding is one of the mainstream approaches for accelerating large model inference, but its efficiency has long been constrained by a structural bottleneck: the draft model must wait for the target model to update its hidden states, forcing drafting and verification to run serially. D3D, proposed by the Fudan University Generative Vision team, attempts to fundamentally eliminate this dependency.

The Serial Bottleneck: The "Invisible Wait" of Diffusion Drafting

Diffusion drafters conditioned on the target model can predict candidate tokens within a block in parallel, appearing far faster than autoregressive drafting. The problem is that generating each new block requires waiting for the target model to return updated hidden states. In other words, although a single block is generated in one parallel forward pass, blocks are forced to be serialized—draft, verify, draft again, verify again—creating strict pipeline stalls.

This explains why diffusion drafting, despite high parallelism on paper, often falls short of expected end-to-end speedups in practice. D3D's starting point is straightforward: if the drafter no longer depends on the target model's hidden states, drafting and verification can truly overlap.

Hidden states refer to the high-dimensional vector representations output by each layer of a Transformer decoder, carrying the model's internal encoding of the current context. In speculative decoding, a single forward pass of the target model produces hidden states for each position, and draft models (especially diffusion-based drafters) often use them as an additional conditioning signal to judge "what tokens the target model will accept." This conditioning can significantly improve acceptance rates, but at the cost of requiring the drafter to wait until the target model finishes computing and returns hidden states before it can begin—creating the "invisible wait" mentioned in the article. D3D's approach essentially abandons this strong conditioning signal and instead compensates drafting quality through other supervision methods.

D3D's Approach: Blockwise Denoising + Auxiliary Autoregressive Supervision

D3D is a standalone diffusion drafter whose core design combines blockwise denoising with auxiliary autoregressive supervision. The former preserves the diffusion model's ability to generate an entire block of candidate tokens in one pass, while the latter maintains long high-quality acceptance prefixes without relying on the target model's hidden states.

The combination compensates for drafting quality: after losing the strong conditioning signal of target hidden states, autoregressive supervision takes on the responsibility of maintaining token coherence, enabling the drafter to still produce sufficiently long candidate sequences that the target model accepts.

This independence unlocks a key capability—D3D can draft subsequent blocks in a single packed forward pass while the target model performs verification, achieving truly asynchronous speculative decoding. The two originally serial stages are compressed into a single timeline.

Acceptance length in speculative decoding refers to the number of consecutive candidate tokens the target model accepts in one draft-verify cycle. During verification, the target model scores the entire drafted block in parallel, comparing from left to right: the first rejected position in the prefix is the truncation point, and the acceptance length is the number of tokens before that point. The longer the acceptance length, the more tokens are produced per verification step on average, and the higher the end-to-end speedup; it is directly determined by how well the drafter's distribution aligns with the target model's. This is why the article repeatedly emphasizes "maintaining long high-quality acceptance prefixes"—this is the core lever of speculative decoding acceleration and the metric used to measure pruning cost during layer pruning.

Layer Pruning: Trading Acceptance Length for Packing Cost

Asynchronous packed drafting is not without cost: processing multiple blocks in one forward pass increases computation. D3D's response is progressive pruning based on each layer's impact on acceptance length, reducing the overhead of packed drafting.

The paper's data is compelling: on three benchmarks with Qwen3-8B as the target model, removing 5 of 36 layers still retains over 99% of average acceptance length on average. That is, nearly 14% of layers can be removed with almost no loss in drafting quality, and this compute savings translates directly into speed.

Diffusion language models (diffusion LMs) as drafters differ from autoregressive drafters: they treat a token block as a noisy sequence and restore the entire block in parallel through multi-step denoising, so generation within a block is naturally parallel. However, diffusion denoising typically requires multiple iterations, and if the input conditions at each step come from the target model, serial dependency is reintroduced. Blockwise denoising restricts the denoising scope to within a single block, making each block's generation a relatively independent parallel forward pass—this is precisely the prerequisite for D3D to "pack" the drafting of multiple blocks into a single forward pass.

Measured Speedup: From 5x to 7.48x

Across seven benchmarks, D3D achieved average decoding speedups of 5.00x and 5.71x with Qwen3-8B and Qwen3-32B as target models, respectively; by comparison, DFlash achieved 4.03x and 3.67x under the same settings. The gap actually widens with larger target models, indicating that the benefits of removing serial dependency scale up with model size.

The extreme case appears on GSM8K: with Qwen3-32B as the target, D3D's speedup reaches 7.48x. Such mathematical reasoning tasks demand high drafting quality, and achieving this figure indirectly confirms the synergistic effect of asynchronous overlap and acceptance length.

The code is open-sourced on GitHub (fudan-generative-vision/D3D).

A Brief Observation

D3D's value lies not in being "yet another faster speculative decoding variant," but in changing the coupling relationship between the drafter and the target model. When the drafter no longer depends on target hidden states, the scheduling space of speculative decoding is significantly broadened—packing, overlapping, and pruning only truly come into play then. For inference infrastructure, this kind of decoupling approach may be more noteworthy than any single-point speedup figure.

Background Supplement

From a broader perspective, the fundamental premise of the "draft-verify" paradigm to which D3D belongs is verification parallelism: a single forward pass of the target model can score the entire block of candidate tokens in parallel, and correction mechanisms such as rejection sampling ensure that the output distribution matches the target model's own sampling without loss of generation quality. This also means that the speedup ceiling of speculative decoding is constrained by two factors—drafting cost must be far lower than a single forward pass of the target model, and acceptance length must be sufficiently long. D3D's asynchronous packing and layer pruning target these two factors respectively: the former amortizes drafting overhead, while the latter preserves acceptance length. Understanding this cost-benefit structure helps assess the applicable boundaries of this approach across different model sizes and task types.