[KongchangAI]
Conceptfrequency-aware remasking

Frequency-Aware Remasking

A decoding strategy for diffusion language models that tracks how frequently each token prediction changes across denoising steps and prioritizes committing high-frequency-change tokens to encourage exploration of alternative reasoning paths and avoid collapse onto a single trajectory.

Timeline (last 90 days)

Nov 10

A paper proposes formalizing the test-time scaling of DLMs as the adaptive allocation of inference compute and decoding strategy, and introduces a frequency-aware remasking strategy.

Unverified50%
Nov 10

This study designs a two-stage method: first use a fast base sampler to solve easy problems, then hand the remaining difficult problems to the frequency-aware remasking strategy.

Unverified50%
Nov 10

Frequency-aware remasking tracks the frequency with which each token's prediction changes across denoising steps, and prioritizes committing those tokens that change frequently.

Unverified50%
Nov 10

Frequency-aware remasking dynamically balances exploration (retaining uncertain tokens to try alternative paths) and exploitation (committing stable tokens to advance generation).

Unverified50%
Nov 10

For problems that the base determinism-aware strategy still failed to solve after 100 samples, frequency-aware remasking was able to recover 3.75% to 18.75% of the cases.

Unverified50%
Nov 10

Frequency-aware remasking provides a lightweight decoding strategy adjustment that improves reasoning capability without retraining the model.

Unverified50%
Nov 10

Frequency-aware remasking is complementary to existing determinism-aware strategies and can be used in combination with them.

Unverified50%

All Facts (7)

Source Articles