NNGP
The function-space distribution of an infinitely wide neural network under specific initialization conditions, which converges to a Gaussian process whose kernel is determined by the network architecture and activation function. The exact Bayesian posterior has an analytic form.
Timeline (last 90 days)
For predictive variance, when c < 1 the variance retains the prior value, when c = 1 the variance matches the NNGP posterior variance, and when c > 1 the variance tends to vanish.
The NNGP posterior refers to the fact that as the neural network width tends to infinity, the distribution of the network in function space converges to a Gaussian process whose kernel function is determined by the network architecture and activation function.
By appropriately choosing τ and c, one can selectively recover either the mean or the variance of the NNGP posterior, but cannot simultaneously recover all statistical properties of the exact posterior without trade-offs.
All Facts (3)
For predictive variance, when c < 1 the variance retains the prior value, when c = 1 the variance matches the NNGP posterior variance, and when c > 1 the variance tends to vanish.
50%UnverifiedThe NNGP posterior refers to the fact that as the neural network width tends to infinity, the distribution of the network in function space converges to a Gaussian process whose kernel function is determined by the network architecture and activation function.
50%UnverifiedBy appropriately choosing τ and c, one can selectively recover either the mean or the variance of the NNGP posterior, but cannot simultaneously recover all statistical properties of the exact posterior without trade-offs.
50%