LMCache
A KV-cache management system for large-model inference that supports multi-tier storage expansion across GPU HBM, CPU DRAM, and SSD, aimed at reducing GPU memory consumption and cost for long-context inference workloads.
Timeline (last 90 days)
Systems such as Mooncake and LMCache are designed around "KV Cache-centric" scheduling, turning cache reuse and tiered offloading into independent services.
Systems such as Mooncake, LMCache, FlexGen, InfiniGen, and AttentionStore attempt to extend GPU VRAM capacity using CPU DRAM and SSDs.
All Facts (2)
Systems such as Mooncake and LMCache are designed around "KV Cache-centric" scheduling, turning cache reuse and tiered offloading into independent services.
50%UnverifiedSystems such as Mooncake, LMCache, FlexGen, InfiniGen, and AttentionStore attempt to extend GPU VRAM capacity using CPU DRAM and SSDs.
50%