ConceptWeight Spectra
Weight Spectrum
The eigenvalue distribution obtained by applying singular value decomposition to a weight matrix. It reflects the spectral properties of the weight matrix and is used to analyze the structural characteristics of model weights.
Timeline (last 90 days)
Oct 15
The weight spectrum refers to the singular value distribution obtained by performing singular value decomposition on the weight matrix, reflecting the energy distribution of the weights in different directions, and can be used to assess the degree to which quantization damages the low-rank structure of the matrix.
Unverified50%