KV Cache Quantization

An optimization technique that compresses KV cache occupancy through quantization, low-rank approximation, and importance-based eviction of old tokens, thereby accommodating longer contexts or higher concurrency within the same GPU memory.