SpecCTMC: Exact Speculative Sampling for Diffusion Language Models

Diffusion Language Models accelerate text generation by generating tokens in parallel, but independently sampling multiple tokens breaks dependencies and changes the output distribution. Although exact continuous-time sampling avoids this approximation, it typically requires one sequential model evaluation per token. This article introduces SpecCTMC, a training-free speculative sampler that reduces this sequential bottleneck without relying on a separate draft model, significantly reducing the number of model evaluations while preserving the original distribution.

The distribution dilemma of parallel generation

The core advantage of Masked Diffusion Language Models lies in parallel decoding: a single forward pass can simultaneously predict tokens at multiple positions. However, this parallelism comes with a statistical cost. When multiple tokens are sampled independently, the dependencies among them are ignored, causing the joint distribution of the generated results to deviate from the distribution originally defined by the model.

This deviation is not trivial. In scenarios requiring strict probabilistic guarantees—such as controllable text generation, scientific computing, or probabilistic programming—distributional fidelity is crucial. The traditional solution is to use exact continuous-time sampling, which simulates the evolution of a continuous-time Markov chain (CTMC), ensuring that the generation order, timing, and values of each token strictly follow the joint distribution of the underlying chain. But the cost is also obvious: exact sampling typically requires one sequential model evaluation per token, completely canceling out the speed advantage of parallel generation.

A continuous-time Markov chain (CTMC) is a probabilistic model that describes random state transitions over continuous time. In masked diffusion language models, a CTMC models text generation as: the process by which each token position jumps from a "masked" state to a specific vocabulary token, with jump times and jump targets determined by a rate matrix. Exact sampling requires strictly simulating this jump process, so each state transition requires recomputing the model output based on the current complete sequence, causing the number of evaluations to be proportional to the number of tokens. Parallel sampling treats jumps at multiple positions as independent events, which amounts to factorizing the joint rate as the product of marginal rates, thereby introducing factorization error. Understanding the rate factorization structure of a CTMC is the key to understanding why SpecCTMC must perform acceptance-correction: only the correction step can "restore" independent proposals to the true transition probabilities of the joint chain.

The three-step mechanism of SpecCTMC

SpecCTMC is designed to break this sequential bottleneck without training an additional draft model. Its core idea is to use the predictions already computed by the model to propose subsequent token updates, thereby parallelizing multiple steps.

Specifically, SpecCTMC operates in three stages. First, it consecutively proposes several token updates based on model predictions already computed in previous steps. These proposals are not random guesses but come from the model's own output distribution, and thus have high acceptance potential. Second, it evaluates the intermediate states of these proposals in parallel rather than sequentially one by one. Finally, it applies an exact acceptance-and-correction procedure to ensure that the final sampled results strictly follow the joint distribution of the original continuous-time Markov chain.

The key to this process is that neither the parallel evaluation nor the correction step introduces time discretization error or additional factorization error. Unlike traditional speculative decoding, SpecCTMC does not require a separate, smaller draft model to generate candidate sequences; it directly uses the main model's own predictions to propose updates, making it a training-free sampling acceleration scheme.

Speculative decoding was originally an acceleration technique for autoregressive language models: a small draft model quickly generates several candidate tokens, which are then verified in parallel by a large model and accepted or rejected, thereby reducing the number of forward passes of the large model while keeping the target distribution unchanged. SpecCTMC borrows the "propose—verify—correct" framework, but the key difference lies in the source of the draft: it does not train an independent draft model, but instead reuses the prediction distribution already computed by the main diffusion model in previous steps as the proposal distribution. This eliminates the distribution mismatch between the draft model and the target model, and also saves additional training and memory overhead. At the same time, because the predictions of diffusion models naturally carry information along the time dimension, proposals can be unfolded continuously along the event-time axis of the CTMC, rather than only extending token by token to the right as in autoregressive speculation.

Theoretical guarantees and experimental validation

The authors of SpecCTMC prove that the sampler preserves the joint distribution of the underlying continuous-time Markov chain over event times, generation order, and token values. This means that, statistically speaking, samples generated by SpecCTMC are indistinguishable from those generated by direct continuous-time sampling. This property is particularly important for applications requiring exact distributional guarantees.

Experiments were conducted on two models, LLaDA-8B-Instruct and Dream-v0-Instruct-7B, covering five benchmarks. The results show that, compared with direct continuous-time sampling, SpecCTMC reduces the number of sequential model evaluations by 1.73 to 4.38 times while maintaining the reference sampling distribution. This speedup is considerable without additional training, indicating that speculative sampling using the model's own predictions is an effective path.

The acceptance-and-correction procedure is the core mechanism ensuring distributional exactness. Its basic flow is: sample a candidate update from the proposal distribution, accept it with a certain probability, and if rejected, resample from the correction distribution. The acceptance probability is designed so that the composite transition probability of "propose—accept/reject—resample" exactly equals the true transition probability of the target CTMC, a construction similar to the Metropolis-Hastings acceptance criterion. SpecCTMC's contribution is to generalize this criterion to continuous time and multi-token parallelism, and to prove that the corrected samples have the same distribution as direct CTMC sampling in all three dimensions: event time, generation order, and token values. In other words, the acceleration does not come from approximation, but from rewriting "serial evaluation" as "parallel proposal plus exact rejection," thereby losing no distributional fidelity mathematically.

Significance for the diffusion language model ecosystem

SpecCTMC resolves a long-standing contradiction for diffusion language models: the conflict between the speed advantage of parallel generation and the sequential evaluation required for exact sampling. By eliminating this bottleneck, it makes diffusion language models more practical in scenarios requiring strict probabilistic guarantees.

Interestingly, SpecCTMC's training-free nature lowers the barrier to adoption. Researchers and developers can directly apply it to existing diffusion language models without retraining or fine-tuning. The code has been open-sourced on GitHub, further promoting reproducibility and follow-up research.

From a broader perspective, SpecCTMC represents one direction in inference optimization for diffusion language models: not changing the model itself, but unleashing parallel potential through smarter sampling algorithms. As diffusion language models continue to improve in scale and quality, such exact and efficient sampling methods will become key infrastructure for their deployment.

Limitations and future directions

SpecCTMC's current method still has some limitations. It relies on the quality of model predictions to generate high-acceptance proposals; if the model is uncertain in some contexts, the acceptance rate may drop and the speedup may weaken. In addition, experiments were validated only on two models and five benchmarks, and performance on a wider range of model architectures and task types remains to be explored.

Future work could explore how to combine SpecCTMC with more efficient model evaluation strategies, such as caching intermediate states or dynamically adjusting proposal length. Extending this method to other types of diffusion models (such as continuous diffusion models) is also an interesting direction.

Overall, SpecCTMC provides a practical and theoretically rigorous acceleration scheme for exact sampling of diffusion language models, offering a new answer to the distributional fidelity problem of parallel text generation.

LLaDA and Dream are currently the most representative open-source masked diffusion language models. LLaDA-8B uses large-scale masked diffusion pretraining and approaches autoregressive models of the same scale on instruction-following tasks; Dream-v0-Instruct-7B explores the instruction fine-tuning path for diffusion models in dialogue scenarios. Both generate text through iterative mask prediction and are naturally suited as testbeds for SpecCTMC. The choice of these two models also illustrates the applicability of the method: it does not depend on a specific architecture, as long as the model can output token distributions at each position and support intermediate state evaluation. However, different models vary greatly in their prediction confidence distributions, which directly affects the proposal acceptance rate—this is also why the authors link acceptance rate to prediction quality and call for validation on a broader range of models.

One-line Summary

SpecCTMC accelerates exact continuous-time sampling for diffusion language models with training-free speculative sampling, reducing evaluations by 1.73–4.38 times.

Paragraph Summary

Diffusion language models accelerate text generation by generating tokens in parallel, but independent sampling breaks token dependencies and changes the output distribution; exact continuous-time sampling preserves fidelity but requires one sequential model evaluation per token. This article introduces SpecCTMC, a training-free speculative sampler: it uses predictions previously computed by the model to consecutively propose multiple token updates, evaluates intermediate states in parallel, and then applies an exact acceptance-correction procedure to ensure the results strictly follow the joint distribution of the original continuous-time Markov chain, without requiring a separate draft model. The authors prove that it preserves the joint distribution over event times, generation order, and token values, and on five benchmarks with LLaDA-8B-Instruct and Dream-v0-Instruct-7B it reduces the number of sequential evaluations by 1.73 to 4.38 times while maintaining the reference distribution.