Fixed-Point Score Matching
A unified training framework for diffusion samplers. By replacing the denoising score identity with a target score identity, the expectation is transferred from being computed under the target distribution to being computed under the current model distribution, forming an iterative training loop. The theoretically optimal velocity field and score field are fixed points of this iteration.
Timeline (last 90 days)
Fixed-point score matching is a unified framework proposed by Dennis Blessing that unifies diffusion sampling algorithms such as marginal shooting, Schrödinger bridge, flow matching, and bridge matching sampling into the same underlying principle.
The core innovation of fixed-point score matching is replacing the denoising score identity in stochastic interpolation with a target score identity, substituting samples generated by the current model for the true target samples, thereby shifting the expectation from computation under the target distribution to computation under the model distribution.
The iterative loop of fixed-point score matching is: generate terminal samples from the current model and store them in a replay buffer, compute the energy gradient at these samples, then supervise learning with standard score matching and flow matching losses, and repeat after updating the model.
The fixed-point score matching framework can derive a target flow matching loss whose minimum is the same as that of the standard flow matching loss, but which depends only on the energy gradient rather than target samples, thereby allowing direct training of a deterministic ODE policy without SDE conversion or adjoint ODE reverse simulation.
On DeepMind Control and Humanoid tasks, the fixed-point score matching method combined with the relevant exploration achieved the best performance among all compared methods.
The fixed-point score matching framework has limitations: the score is inherently a local quantity, and when the modes of the target distribution are far apart, the method struggles to correctly weigh the relative weights of the modes.
The uniqueness and convergence of the fixed-point iteration have not yet been proven.
All Facts (7)
Fixed-point score matching is a unified framework proposed by Dennis Blessing that unifies diffusion sampling algorithms such as marginal shooting, Schrödinger bridge, flow matching, and bridge matching sampling into the same underlying principle.
50%UnverifiedThe core innovation of fixed-point score matching is replacing the denoising score identity in stochastic interpolation with a target score identity, substituting samples generated by the current model for the true target samples, thereby shifting the expectation from computation under the target distribution to computation under the model distribution.
50%UnverifiedThe iterative loop of fixed-point score matching is: generate terminal samples from the current model and store them in a replay buffer, compute the energy gradient at these samples, then supervise learning with standard score matching and flow matching losses, and repeat after updating the model.
50%UnverifiedThe fixed-point score matching framework can derive a target flow matching loss whose minimum is the same as that of the standard flow matching loss, but which depends only on the energy gradient rather than target samples, thereby allowing direct training of a deterministic ODE policy without SDE conversion or adjoint ODE reverse simulation.
50%UnverifiedOn DeepMind Control and Humanoid tasks, the fixed-point score matching method combined with the relevant exploration achieved the best performance among all compared methods.
50%UnverifiedThe fixed-point score matching framework has limitations: the score is inherently a local quantity, and when the modes of the target distribution are far apart, the method struggles to correctly weigh the relative weights of the modes.
50%UnverifiedThe uniqueness and convergence of the fixed-point iteration have not yet been proven.
50%