Low-Rank KV Approximation

A large model inference optimization technique that exploits the low-rank structure of attention matrices. Methods such as singular value decomposition compress the Key-Value cache by replacing full KV vectors with low-dimensional projections, accelerating the prefill stage and reducing GPU memory consumption.