Low-Rank Compression
A model compression technique that decomposes the weight matrices of neural network linear layers into the product of two low-rank matrices, typically via SVD, to reduce parameter count and computational cost, thereby lowering the overhead of deploying large model inference.
Timeline (last 90 days)
Traditional low-rank compression uses fixed rank allocation during inference, investing the same computational resources regardless of the complexity of input tokens.
Low-rank decomposition is typically implemented using Singular Value Decomposition (SVD), reducing the parameter count from m×n to r×(m+n).
The choice of rank r is a trade-off between accuracy and efficiency: the smaller the r, the higher the compression ratio but the larger the approximation error.
All Facts (3)
Traditional low-rank compression uses fixed rank allocation during inference, investing the same computational resources regardless of the complexity of input tokens.
50%UnverifiedLow-rank decomposition is typically implemented using Singular Value Decomposition (SVD), reducing the parameter count from m×n to r×(m+n).
50%UnverifiedThe choice of rank r is a trade-off between accuracy and efficiency: the smaller the r, the higher the compression ratio but the larger the approximation error.
50%