Marginal Likelihood
A central quantity in Bayesian statistics that measures how well a model explains observed data. Defined as the probability of the data integrated over all possible parameter values, it naturally penalizes model complexity, embodying the Occam's razor effect. It is commonly used for Bayesian model comparison and selection.
Timeline (last 90 days)
The study derives the marginal likelihood of a DNN using a probabilistic framework, and this quantity can be evaluated using only the weights of the network's last layer.
Evaluating the marginal likelihood using only the last-layer weights requires no access to training data.
The marginal likelihood in Bayesian statistics naturally includes a penalty for model complexity, i.e., the Occam's razor effect.
The research team validated on large-scale convolutional networks and large language models that the marginal likelihood can predict model quality, spanning different architectures and task scales.
Since the metric depends only on the weight matrix, it can provide a model evaluation pathway for scenarios where training data is inaccessible.
All Facts (5)
The study derives the marginal likelihood of a DNN using a probabilistic framework, and this quantity can be evaluated using only the weights of the network's last layer.
50%UnverifiedEvaluating the marginal likelihood using only the last-layer weights requires no access to training data.
50%UnverifiedThe marginal likelihood in Bayesian statistics naturally includes a penalty for model complexity, i.e., the Occam's razor effect.
50%UnverifiedThe research team validated on large-scale convolutional networks and large language models that the marginal likelihood can predict model quality, spanning different architectures and task scales.
50%UnverifiedSince the metric depends only on the weight matrix, it can provide a model evaluation pathway for scenarios where training data is inaccessible.
50%