Conjugate Moment Measures
A theoretical framework for generative models that simultaneously characterizes the sampling distribution and transport map through a single convex potential function. It combines moment measure decomposition with optimal transport theory, addressing the mismatch between the prior moment measure and the target distribution in the original moment measure formulation.
Timeline (last 90 days)
The conjugate moment measure proposes using a single convex potential function W to simultaneously characterize the sampling distribution and the transport map, offering a path for generative modeling that differs from mainstream diffusion models.
The conjugate moment measure differs from the original decomposition solely in that the transport uses the gradient of the Gaussian transform of the potential function, rather than the gradient of the potential function directly.
The author proves the existence of a solution to the conjugate moment measure decomposition equation using Schauder's fixed-point theorem, with the proof requiring compactness assumptions (ρ absolutely continuous and supported on a compact convex set).
When ρ has covariance Σ, the prior Gibbs distribution corresponding to the conjugate decomposition has covariance Σ^(1/3), causing the prior to also be dispersed when ρ is dispersed, resulting in better alignment between the two and preventing the Lipschitz constant from exploding.
In practice, it was found that even when only one gradient update is performed on the target per step—yielding a noisy estimate of the optimal transport map—the algorithm is still able to converge.
During sampling, Langevin Monte Carlo is used to draw samples from the log-concave distribution, with the step size adaptively adjusted according to the maximum eigenvalue of the Hessian of W.
All Facts (6)
The conjugate moment measure proposes using a single convex potential function W to simultaneously characterize the sampling distribution and the transport map, offering a path for generative modeling that differs from mainstream diffusion models.
50%UnverifiedThe conjugate moment measure differs from the original decomposition solely in that the transport uses the gradient of the Gaussian transform of the potential function, rather than the gradient of the potential function directly.
50%UnverifiedThe author proves the existence of a solution to the conjugate moment measure decomposition equation using Schauder's fixed-point theorem, with the proof requiring compactness assumptions (ρ absolutely continuous and supported on a compact convex set).
50%UnverifiedWhen ρ has covariance Σ, the prior Gibbs distribution corresponding to the conjugate decomposition has covariance Σ^(1/3), causing the prior to also be dispersed when ρ is dispersed, resulting in better alignment between the two and preventing the Lipschitz constant from exploding.
50%UnverifiedIn practice, it was found that even when only one gradient update is performed on the target per step—yielding a noisy estimate of the optimal transport map—the algorithm is still able to converge.
50%UnverifiedDuring sampling, Langevin Monte Carlo is used to draw samples from the log-concave distribution, with the step size adaptively adjusted according to the maximum eigenvalue of the Hessian of W.
50%