Annealed Lévy Diffusion
A heavy-tailed diffusion model framework that anneals the stability index over time, using heavy-tailed noise for exploration in the early stages of diffusion and approaching Gaussian noise for refinement in later stages, balancing exploration and reconstruction accuracy.
Timeline (last 90 days)
Annealed Lévy Diffusion (ALD) turns the stability index α(t) from a fixed value into an annealing function that varies with time t, using heavier-tailed characteristics early on and approaching Gaussian later.
ALD uses a marginal preferential decomposition, expressing stable noise in the form of a scalar Gaussian scale mixture, so that the training objective returns to the standard Gaussian noise prediction loss, requiring only one additional scalar scale input.
The paper evaluates ALD on four 2D benchmark datasets and an imbalanced nine-peak mixture distribution, and examines training efficiency as well as the effect of the predicted perturbations on clipped Holtsmark data.
ALD's generation quality is validated on MNIST and CelebA64, and it is used for the data augmentation task on CIFAR-10LT.
The ALD paper claims that ALD is able to preserve local geometric properties on the light-tailed targets and manifold-structured targets evaluated.
In ALD's annealing strategy, the schedule design of the hyperparameter α(t), the performance of different samplers on large-scale high-resolution tasks, and the impact of heavy-tailed noise on training stability remain questions requiring further verification.
The ALD paper currently focuses mainly on small- and medium-scale benchmarks, and its scalability to larger-scale generation tasks remains to be tested by follow-up work.
All Facts (7)
Annealed Lévy Diffusion (ALD) turns the stability index α(t) from a fixed value into an annealing function that varies with time t, using heavier-tailed characteristics early on and approaching Gaussian later.
50%UnverifiedALD uses a marginal preferential decomposition, expressing stable noise in the form of a scalar Gaussian scale mixture, so that the training objective returns to the standard Gaussian noise prediction loss, requiring only one additional scalar scale input.
50%UnverifiedThe paper evaluates ALD on four 2D benchmark datasets and an imbalanced nine-peak mixture distribution, and examines training efficiency as well as the effect of the predicted perturbations on clipped Holtsmark data.
50%UnverifiedALD's generation quality is validated on MNIST and CelebA64, and it is used for the data augmentation task on CIFAR-10LT.
50%UnverifiedThe ALD paper claims that ALD is able to preserve local geometric properties on the light-tailed targets and manifold-structured targets evaluated.
50%UnverifiedIn ALD's annealing strategy, the schedule design of the hyperparameter α(t), the performance of different samplers on large-scale high-resolution tasks, and the impact of heavy-tailed noise on training stability remain questions requiring further verification.
50%UnverifiedThe ALD paper currently focuses mainly on small- and medium-scale benchmarks, and its scalability to larger-scale generation tasks remains to be tested by follow-up work.
50%