Product

InfiniGen

A KV-cache management system for large-model inference that supports offloading KV caches to CPU DRAM and SSD, using predictive prefetching to improve efficiency for long-context inference.

Timeline (last 90 days)

Sep 16

Systems such as Mooncake, LMCache, FlexGen, InfiniGen, and AttentionStore attempt to extend GPU VRAM capacity using CPU DRAM and SSDs.

Unverified50%

All Facts (1)

Unverified

Systems such as Mooncake, LMCache, FlexGen, InfiniGen, and AttentionStore attempt to extend GPU VRAM capacity using CPU DRAM and SSDs.

50%

Source Articles

Where Should KV Cache Live? A Deep Dive into GPU/CPU/SSD Tiered Placement StrategiesSep 16