Energy-Based Model
A class of machine learning models that assign a scalar energy value to each configuration in the input space. Low energy corresponds to highly compatible input combinations. Long advocated by Yann LeCun, these models are used to represent constraints on the state of the world, capturing dependencies between observed variables without requiring explicit probabilistic normalization.
Timeline (last 90 days)
The Stein identity constructs an expectation expression that does not depend on the normalization constant; even if the normalization constant Z is unknown, the score function can still be computed, which makes Stein methods suitable for scenarios requiring inference on unnormalized distributions (such as Bayesian posteriors and energy models).
Around 2006, LeCun and others systematically organized the Energy-Based Model (EBM) framework.
The computational complexity of modeling interactions between vectors as attractive/repulsive forces typically grows quadratically with the size of the candidate set.
Energy-Based Models (EBMs) are cheap to train but expensive to sample from, because each sample requires running a complete MCMC chain, with no amortization of the sampling cost.
Mattick believes that Energy-Based Models are no longer truly useful: if the goal is generation, diffusion models or flow matching are better; if the goal is self-supervised learning, Energy-Based Models are too broad and lack informativeness.
Yann LeCun advocates for Energy-Based Models and argues they are superior because they are 'not generative,' while Mattick counters that Energy-Based Models are fundamentally still generative.
All Facts (7)
The Stein identity constructs an expectation expression that does not depend on the normalization constant; even if the normalization constant Z is unknown, the score function can still be computed, which makes Stein methods suitable for scenarios requiring inference on unnormalized distributions (such as Bayesian posteriors and energy models).
50%UnverifiedAround 2006, LeCun and others systematically organized the Energy-Based Model (EBM) framework.
50%UnverifiedThe computational complexity of modeling interactions between vectors as attractive/repulsive forces typically grows quadratically with the size of the candidate set.
50%UnverifiedYann LeCun advocates for Energy-Based Models and argues they are superior because they are 'not generative,' while Mattick counters that Energy-Based Models are fundamentally still generative.
50%UnverifiedEnergy-Based Models (EBMs) are cheap to train but expensive to sample from, because each sample requires running a complete MCMC chain, with no amortization of the sampling cost.
50%UnverifiedMattick believes that Energy-Based Models are no longer truly useful: if the goal is generation, diffusion models or flow matching are better; if the goal is self-supervised learning, Energy-Based Models are too broad and lack informativeness.
50%Unverified能量模型（EBM）的思想是为输入空间中的每个配置分配一个标量能量值，低能量对应高兼容性的输入组合
50%