Masked Diffusion
A generative paradigm in which training randomly masks some tokens in a sequence and learns to restore them; at generation time the full sequence is recovered from a masked state through multiple rounds of iterative denoising, allowing parallel prediction of multiple positions.
Timeline (last 90 days)
Masked diffusion randomly masks some tokens in the sequence during training, and during generation starts from the masked state and gradually denoises through multiple iterations to restore the complete sequence, with each step able to predict tokens at multiple positions in parallel.
Traditional discrete diffusion (masked diffusion and uniform diffusion) completely erases token identity in a single corruption event, so the noisy state cannot express partial uncertainty (e.g., multiple candidates still being possible).
Under matched architectures and controlled training/sampling/tuning budgets, the symmetric form of binary diffusion consistently outperformed finely tuned masked diffusion and uniform diffusion baselines.
The MANNERS project handles the semi-grand canonical ensemble, jointly modeling continuous channels (atomic positions and volume) and discrete channels (atomic species), with the discrete part using a masked diffusion model, and derives a label cross-entropy loss whose conditional expectation corresponds exactly to the true denoising posterior.
All Facts (4)
Masked diffusion randomly masks some tokens in the sequence during training, and during generation starts from the masked state and gradually denoises through multiple iterations to restore the complete sequence, with each step able to predict tokens at multiple positions in parallel.
50%UnverifiedTraditional discrete diffusion (masked diffusion and uniform diffusion) completely erases token identity in a single corruption event, so the noisy state cannot express partial uncertainty (e.g., multiple candidates still being possible).
50%UnverifiedUnder matched architectures and controlled training/sampling/tuning budgets, the symmetric form of binary diffusion consistently outperformed finely tuned masked diffusion and uniform diffusion baselines.
50%UnverifiedThe MANNERS project handles the semi-grand canonical ensemble, jointly modeling continuous channels (atomic positions and volume) and discrete channels (atomic species), with the discrete part using a masked diffusion model, and derives a label cross-entropy loss whose conditional expectation corresponds exactly to the true denoising posterior.
50%