Remasking
An operation in the iterative denoising process of diffusion language models that re-marks committed tokens as undetermined so they have a chance to be re-predicted, used to adjust the generation trajectory.
Timeline (last 90 days)
Remasking methods require additional forward evaluations to progressively replace tokens, incurring substantial computational overhead.
On the Sudoku task, when re-masking an isolated error while keeping the surrounding context unchanged, the regenerated result is often the same error again.
Remasking refers to re-marking certain already-committed tokens as undetermined before the next denoising step, giving them a chance to be re-predicted.
Continuous diffusion language models map discrete tokens to a continuous embedding space, perform the noising and denoising process in that space, and finally decode back to discrete tokens.
All Facts (4)
Remasking methods require additional forward evaluations to progressively replace tokens, incurring substantial computational overhead.
50%UnverifiedOn the Sudoku task, when re-masking an isolated error while keeping the surrounding context unchanged, the regenerated result is often the same error again.
50%UnverifiedRemasking refers to re-marking certain already-committed tokens as undetermined before the next denoising step, giving them a chance to be re-predicted.
50%UnverifiedContinuous diffusion language models map discrete tokens to a continuous embedding space, perform the noising and denoising process in that space, and finally decode back to discrete tokens.
50%