Low-rank approximation
A mathematical method that approximates a matrix using a decomposition of rank far smaller than the original matrix dimension, greatly reducing computational cost. A common form is H ≈ U·M·U^T.
Timeline (last 90 days)
Compared with the full-rank (no low-rank approximation) approach, VPU at dim=256 reduced 200 steps from 4249ms to 160ms, achieving a 26.44x speedup.
The mathematical foundation of low-rank approximation comes from singular value decomposition (SVD) in linear algebra, which can approximate a matrix with a small effective rank using the product of two low-dimensional matrices U·V^T.
Architecture-level optimizations (such as GQA) and inference-time optimizations (such as low-rank approximation and quantization) are orthogonal and can be used in combination.
All Facts (3)
Compared with the full-rank (no low-rank approximation) approach, VPU at dim=256 reduced 200 steps from 4249ms to 160ms, achieving a 26.44x speedup.
50%UnverifiedThe mathematical foundation of low-rank approximation comes from singular value decomposition (SVD) in linear algebra, which can approximate a matrix with a small effective rank using the product of two low-dimensional matrices U·V^T.
50%UnverifiedArchitecture-level optimizations (such as GQA) and inference-time optimizations (such as low-rank approximation and quantization) are orthogonal and can be used in combination.
50%