Spectral Bias
A phenomenon in neural network training whereby the convergence speed of output error systematically decreases as frequency increases across frequency components. That is, the network preferentially learns low-frequency smooth patterns and fits high-frequency sharp variations later. It is a key clue for understanding generalization and implicit regularization.
Timeline (last 90 days)
During neural network training, low-frequency, smooth patterns are learned first, followed by fitting high-frequency, sharp variations; this phenomenon is called spectral bias or the F-principle.
The spectral bias phenomenon was first systematically characterized by Xu et al. around 2019 through a large number of numerical experiments.
The 2019 work by Rahaman et al. provided an approximate theoretical explanation of spectral bias.
All Facts (3)
During neural network training, low-frequency, smooth patterns are learned first, followed by fitting high-frequency, sharp variations; this phenomenon is called spectral bias or the F-principle.
50%UnverifiedThe spectral bias phenomenon was first systematically characterized by Xu et al. around 2019 through a large number of numerical experiments.
50%UnverifiedThe 2019 work by Rahaman et al. provided an approximate theoretical explanation of spectral bias.
50%