[KongchangAI]
· 1 min read· 961 words

When Should Diffusion Language Models Regenerate? The DiVR Verifier Answers

When Should Diffusion Language Models Regenerate? The DiVR Verifier Answers

DiVR verifier guides Diffusion Language Model remasking error correction by predicting final sequence correctness, significantly improving inference accuracy.

Diffusion Language Models generate text through iterative denoising and can correct errors via remasking, but existing methods face two challenges: error information in hidden states cannot be reliably exposed by token confidence, and isolated remasking tends to repeat the same errors. To address this, researchers proposed the DiVR verifier, trained on the final correctness of the model's own trajectories to predict whether a partially generated sequence can reach the correct answer, thereby deciding when to regenerate and which candidate to retain. On five reasoning benchmarks, DiVR improved LLaDA-8B-Base's average accuracy from 35.3% to 43.4% at only 1.9–3.5× the cost of greedy decoding, outperforming more expensive search baselines. DiVR is also complementary to SFT and RL, and generalizes to other generators such as Dream.

Diffusion Language Models generate text through iterative denoising. Their unique feature is the ability to "remask" already-generated tokens and regenerate them, thereby correcting errors. However, determining when to regenerate and which candidate to retain has long puzzled researchers. A recent study diagnosed the Sudoku task, revealing two key gaps in existing methods, and proposed the DiVR verifier, which enables the model to learn "when to start over" during generation.

The Remasking Dilemma: Error Information Hidden in Hidden States

The research team conducted a detailed analysis of the Diffusion Language Model generation process on the Sudoku task, uncovering two issues. First, the model's hidden states contain error information, but this information cannot be reliably exposed by token confidence. In other words, the model may exhibit high confidence in an erroneous token, causing confidence-based remasking strategies to fail. Second, when the model remasks an isolated error while the surrounding context remains unchanged, the regenerated result is often the same error. This indicates that relying solely on local remasking makes effective correction difficult; the model needs more global judgment.

These two findings point to the same conclusion: effective error correction requires not only knowing "where errors might be" but also predicting "whether the entire sequence will ultimately be correct."

Diffusion Language Models borrow the idea of diffusion models from image generation, treating text generation as an iterative denoising process: starting from a fully masked sequence, tokens are progressively predicted and filled in until complete text is formed. Unlike autoregressive models that generate token by token from left to right, Diffusion Language Models can predict tokens at multiple positions in parallel and modify already-generated content in subsequent steps. This gives them a unique "remasking" capability—re-marking generated tokens as masked and regenerating them. Theoretically, this capability allows the model to correct early errors, but how to effectively leverage it remains an open question.

DiVR: A Verifier that Predicts Final Correctness

Based on the above diagnosis, the researchers proposed DiVR (Diffusion Verifier), an independent verifier that predicts whether a partially generated sequence will ultimately reach the correct answer. DiVR's training data comes from the final correctness of the model's own trajectories—that is, letting the model generate a large number of trajectories, recording whether each trajectory ultimately succeeds, and then training the verifier on these results.

At inference time, DiVR decides when to regenerate a block and which candidate to retain. Interestingly, the generator itself remains unchanged; DiVR works as an external module, making it plug-and-play. This design allows DiVR to be compatible with different generators without retraining the generative model.

Verifiers in machine learning typically refer to a model independent of the generator, used to evaluate the quality or correctness of candidate outputs. In language models, verifiers are often used for reranking or search guidance, such as selecting the most likely correct answer from multiple candidates. DiVR's innovation lies in applying the verifier to the iterative generation process of Diffusion Language Models: it does not directly generate tokens but predicts whether the current partially generated sequence will ultimately reach the correct answer, thereby guiding regeneration decisions. This decoupled "generate-verify" design avoids modifying the generator itself, making the verifier plug-and-play.

Experimental Results: Accuracy Gains and Cost Trade-offs

The researchers conducted experiments using LLaDA-8B-Base on five reasoning benchmarks. Results show that DiVR improved the average accuracy of the fine-tuned generator from 35.3% to 43.4%, at a cost of 1.9–3.5× that of greedy decoding. In contrast, search-based baseline methods, despite using 3.8–7.4× the cost, still achieved lower accuracy than DiVR. This indicates that DiVR achieves a better balance between cost and performance.

Furthermore, DiVR is complementary to post-training methods such as supervised fine-tuning (SFT) and reinforcement learning (RL), and can generalize to other generators, including Dream. This further validates its generality.

Greedy Decoding is the simplest strategy in language model inference: at each step, select the token with the highest probability, without backtracking or sampling. In Diffusion Language Models, greedy decoding typically means filling in the highest-confidence predicted token at each step until the sequence is complete. It has the lowest computational cost but easily falls into local optima and cannot correct early errors. DiVR trades 1.9–3.5× the cost of greedy decoding for accuracy gains, with the cost mainly arising from multiple verifier calls and regeneration operations. In contrast, search-based baselines (such as beam search) need to evaluate more candidate sequences, incurring higher costs with limited benefits.

Implications and Outlook

The core insight of DiVR is that error correction in Diffusion Language Models should not rely solely on token-level confidence but should be verified from the perspective of sequence-level correctness. This decoupled "generate-verify" approach provides a new direction for improving the inference reliability of Diffusion Language Models. In the future, how to integrate the verifier more tightly with the generator and how to reduce verification costs may be worthwhile questions to explore.

Illustration of Diffusion Language Model regeneration verifier

One-line Summary

DiVR verifier guides Diffusion Language Model remasking error correction by predicting final sequence correctness, significantly improving inference accuracy.

Paragraph Summary

Diffusion Language Models generate text through iterative denoising and can correct errors via remasking, but existing methods face two challenges: error information in hidden states cannot be reliably exposed by token confidence, and isolated remasking tends to repeat the same errors. To address this, researchers proposed the DiVR verifier, trained on the final correctness of the model's own trajectories to predict whether a partially generated sequence can reach the correct answer, thereby deciding when to regenerate and which candidate to retain. On five reasoning benchmarks, DiVR improved LLaDA-8B-Base's average accuracy from 35.3% to 43.4% at only 1.9–3.5× the cost of greedy decoding, outperforming more expensive search baselines. DiVR is also complementary to SFT and RL, and generalizes to other generators such as Dream.

#Diffusion Language Model#DiVR verifier#remasking#LLaDA-8B-Base#greedy decoding#sequence-level correctness
Share:

Related articles