SpecCTMC
A training-free exact speculative sampler for diffusion language models. It reuses the model's own predictions as proposals and applies an acceptance-correction procedure, reducing the number of sequential model evaluations while preserving the original CTMC joint distribution.
Timeline (last 90 days)
SpecCTMC is a training-free speculative sampler for reducing the sequential model-evaluation bottleneck in exact continuous-time sampling of diffusion language models.
SpecCTMC uses the predictive distributions already computed in previous steps of the model to continuously propose multiple token updates and evaluates the intermediate states in parallel.
SpecCTMC employs an exact accept-correct procedure to ensure that the final sampling results strictly follow the joint distribution of the original continuous-time Markov chain.
Unlike traditional speculative decoding, SpecCTMC does not require a separate draft model and instead directly reuses the main diffusion model's own predictive distributions as the proposal distribution.
Compared with direct continuous-time sampling, SpecCTMC reduces the number of sequential model evaluations by 1.73 to 4.38 times while maintaining the reference sampling distribution.
SpecCTMC's accept-correct procedure resembles the Metropolis-Hastings acceptance criterion, generalizing the single-token criterion to continuous-time, multi-token parallel settings.
SpecCTMC relies on the quality of model predictions to generate proposals with high acceptance rates; if model predictions are uncertain, acceptance rates may drop and the speedup may diminish.
SpecCTMC's experiments were validated on only two models and five benchmarks, and performance on a broader range of model architectures and task types remains to be explored.
All Facts (8)
SpecCTMC is a training-free speculative sampler for reducing the sequential model-evaluation bottleneck in exact continuous-time sampling of diffusion language models.
50%UnverifiedSpecCTMC uses the predictive distributions already computed in previous steps of the model to continuously propose multiple token updates and evaluates the intermediate states in parallel.
50%UnverifiedSpecCTMC employs an exact accept-correct procedure to ensure that the final sampling results strictly follow the joint distribution of the original continuous-time Markov chain.
50%UnverifiedUnlike traditional speculative decoding, SpecCTMC does not require a separate draft model and instead directly reuses the main diffusion model's own predictive distributions as the proposal distribution.
50%UnverifiedCompared with direct continuous-time sampling, SpecCTMC reduces the number of sequential model evaluations by 1.73 to 4.38 times while maintaining the reference sampling distribution.
50%UnverifiedSpecCTMC's accept-correct procedure resembles the Metropolis-Hastings acceptance criterion, generalizing the single-token criterion to continuous-time, multi-token parallel settings.
50%UnverifiedSpecCTMC relies on the quality of model predictions to generate proposals with high acceptance rates; if model predictions are uncertain, acceptance rates may drop and the speedup may diminish.
50%UnverifiedSpecCTMC's experiments were validated on only two models and five benchmarks, and performance on a broader range of model architectures and task types remains to be explored.
50%