Xavier Initialization
A classical neural network weight initialization method that sets weight variance proportional to the reciprocal of the sum of input and output dimensions, keeping signal variance stable during forward and backward propagation. It is suited to symmetric linear activation functions such as tanh and sigmoid.
Timeline (last 90 days)
Xavier initialization (Glorot, 2010) and He initialization (He et al., 2015) are based on Gaussian distributions and ensure that the output variance of each layer is consistent with the input by designing the variance to be 1/n and 2/n respectively, where n is the input dimension.
Xavier initialization (Glorot, 2010) and He initialization (He et al., 2015) are based on Gaussian or uniform distributions and maintain stable gradient flow by designing variances of 1/n and 2/n respectively (where n is the input dimension).
Xavier initialization was proposed by Xavier Glorot in 2010, sampling weights from a distribution with mean 0 and variance 2/(n_in + n_out), and is suitable for symmetric activation functions such as Sigmoid/Tanh.
All Facts (3)
Xavier initialization (Glorot, 2010) and He initialization (He et al., 2015) are based on Gaussian distributions and ensure that the output variance of each layer is consistent with the input by designing the variance to be 1/n and 2/n respectively, where n is the input dimension.
50%UnverifiedXavier initialization (Glorot, 2010) and He initialization (He et al., 2015) are based on Gaussian or uniform distributions and maintain stable gradient flow by designing variances of 1/n and 2/n respectively (where n is the input dimension).
50%UnverifiedXavier initialization was proposed by Xavier Glorot in 2010, sampling weights from a distribution with mean 0 and variance 2/(n_in + n_out), and is suitable for symmetric activation functions such as Sigmoid/Tanh.
50%