SMD
A few-step diffusion and flow-matching distillation method that achieves 4-step generation using only online velocity regression plus a lightweight residual surrogate called Pivot. It discards auxiliary score models and adversarial training, and introduces no additional computation graph changes at deployment time.
Timeline (last 90 days)
Simplified Matching Distillation (SMD) is a few-step distillation method whose core claim is that simple on-policy velocity regression alone is sufficient to support 4-NFE generation quality.
SMD introduces a residual surrogate called Pivot, whose role is to separate local corrections from the student's interval-mean prediction: the student is responsible for predicting the average velocity over the entire interval, and Pivot is responsible for modeling the residual fluctuations within the interval.
SMD also adds a repulsion mechanism: keeping the student away from the teacher's weaker predictions, to avoid inheriting the teacher's unreliable prediction patterns at certain time steps.
SMD's repulsion mechanism does not require the discriminator commonly used in adversarial training, nor does it require online score estimation; it is merely an additional constraint on the training objective, and the training overhead is almost zero.
SMD's experiments cover image generation and editing, 3D generation, video synthesis, and joint audio-video generation, with model scale extended up to 33B parameters.
Under the 4-NFE setting, SMD surpasses existing strong few-step baselines on multiple benchmarks, and even outperforms the multi-step teacher model on some tasks.
SMD supports cross-model distillation, multi-teacher distillation, and causal video distillation, and is not tied to any particular teacher architecture or modality.
All Facts (7)
Simplified Matching Distillation (SMD) is a few-step distillation method whose core claim is that simple on-policy velocity regression alone is sufficient to support 4-NFE generation quality.
50%UnverifiedSMD introduces a residual surrogate called Pivot, whose role is to separate local corrections from the student's interval-mean prediction: the student is responsible for predicting the average velocity over the entire interval, and Pivot is responsible for modeling the residual fluctuations within the interval.
50%UnverifiedSMD also adds a repulsion mechanism: keeping the student away from the teacher's weaker predictions, to avoid inheriting the teacher's unreliable prediction patterns at certain time steps.
50%UnverifiedSMD's repulsion mechanism does not require the discriminator commonly used in adversarial training, nor does it require online score estimation; it is merely an additional constraint on the training objective, and the training overhead is almost zero.
50%UnverifiedSMD's experiments cover image generation and editing, 3D generation, video synthesis, and joint audio-video generation, with model scale extended up to 33B parameters.
50%UnverifiedUnder the 4-NFE setting, SMD surpasses existing strong few-step baselines on multiple benchmarks, and even outperforms the multi-step teacher model on some tasks.
50%UnverifiedSMD supports cross-model distillation, multi-teacher distillation, and causal video distillation, and is not tied to any particular teacher architecture or modality.
50%