SEDD
SEDD (Score Entropy Discrete Diffusion) is a representative work in discrete masked diffusion language modeling that performs language generation in discrete token spaces using a score entropy-based discrete diffusion approach.
Core Facts
Timeline (last 90 days)
Mainstream samplers such as MDLM and SEDD rely on confidence scores from the current timestep to make greedy decisions when selecting fixed positions, and once a position is decoded it no longer participates in subsequent denoising cycles.
MDLM and SEDD are representative works in discrete diffusion models.
In arena evaluations on continuation benchmarks, Reviser significantly outperforms two discrete diffusion/masked language model baselines, SEDD and MDLM.
Discrete masked diffusion draws on BERT's masked language modeling idea—randomly masking and restoring sequences during training and iteratively filling in from a fully masked sequence at inference—with representative works including MDLM and SEDD.
离散扩散模型发展经历D3PM（2021年Austin等人）、MDLM（2023年Sahoo等人）、SEDD（2023年Lou等人）等里程碑，但模型规模通常不超过1B参数
All Facts (7)
扩散模型的核心原理分为前向扩散过程（逐步添加高斯噪声）和反向去噪过程（从噪声中还原图像）两个阶段
90%Unverified早期的离散扩散语言模型如 MDLM、SEDD 通过在 Token 嵌入空间或掩码空间中定义扩散过程来解决文本离散性挑战
85%UnverifiedMainstream samplers such as MDLM and SEDD rely on confidence scores from the current timestep to make greedy decisions when selecting fixed positions, and once a position is decoded it no longer participates in subsequent denoising cycles.
50%UnverifiedMDLM and SEDD are representative works in discrete diffusion models.
50%UnverifiedIn arena evaluations on continuation benchmarks, Reviser significantly outperforms two discrete diffusion/masked language model baselines, SEDD and MDLM.
50%UnverifiedDiscrete masked diffusion draws on BERT's masked language modeling idea—randomly masking and restoring sequences during training and iteratively filling in from a fully masked sequence at inference—with representative works including MDLM and SEDD.
50%Unverified离散扩散模型发展经历D3PM（2021年Austin等人）、MDLM（2023年Sahoo等人）、SEDD（2023年Lou等人）等里程碑，但模型规模通常不超过1B参数
50%