ConceptLow-Rank Decomposition
Low-Rank Decomposition
A model compression technique that decomposes a large matrix into the product of several smaller matrices to reduce parameter count and computation. It is commonly used to perform low-rank approximation of the recurrent state in state space models.
Timeline (last 90 days)
Oct 16
Quantization, pruning, and low-rank decomposition are three common means of model compression.
Unverified50%
Oct 9
Low-rank decomposition introduces truncation errors that degrade model accuracy, which must be compensated through brief fine-tuning (referred to as healing/repair).
Unverified50%