DMD
A distillation technique for few-step generative models that trains a student model by minimizing the divergence between the student and teacher distributions, commonly used to accelerate sampling in diffusion models.
Timeline (last 90 days)
Distribution Matching Distillation (DMD) trains the student model via the difference between the teacher model's score and the online critic's score of the student distribution.
In conventional DMD training, the critic is required to model a step-independent mixture distribution of student outputs.
The critic's capacity and conditional information design are a general bottleneck of DMD distillation, rather than an incidental phenomenon specific to a particular task.
DMD (Distribution Matching Distillation) is an acceleration technique that distills a multi-step diffusion model into a generator with very few steps (typically 4-8 steps).
All Facts (4)
Distribution Matching Distillation (DMD) trains the student model via the difference between the teacher model's score and the online critic's score of the student distribution.
50%UnverifiedIn conventional DMD training, the critic is required to model a step-independent mixture distribution of student outputs.
50%UnverifiedThe critic's capacity and conditional information design are a general bottleneck of DMD distillation, rather than an incidental phenomenon specific to a particular task.
50%UnverifiedDMD (Distribution Matching Distillation) is an acceleration technique that distills a multi-step diffusion model into a generator with very few steps (typically 4-8 steps).
50%