Heavy-Tailed Weight Spectral Distribution

A phenomenon in which the singular value or eigenvalue distributions of deep neural network weight matrices exhibit heavy-tailed characteristics. Empirical studies indicate a positive correlation between this property and model generalization quality, suggesting it can serve as a proxy indicator of model quality without requiring access to training data.