LRU
Least Recently Used, a cache eviction policy that prioritizes evicting the least recently used data, employed to balance memory usage and cache hit rate.
Timeline (last 90 days)
Cache eviction policies include LRU, LFU, and weighted schemes that consider prefix length or recomputation cost; frequent cache eviction reduces the effectiveness of cache-aware routing.
GPU expert weight caching typically uses a replacement strategy similar to LRU (Least Recently Used), prioritizing retention of expert weights that have been frequently invoked recently.
LRU is far harder to beat than KV-Cache papers suggest.
A large body of research has attempted to design smarter cache eviction strategies than LRU, such as deciding which cache entries to retain based on token importance, attention scores, or access frequency.
Advanced caching strategies require additional metadata tracking, scoring computation, and maintenance costs, and after accounting for these overheads, their net gain relative to LRU may shrink substantially.
LRU is extremely lightweight to implement and performs stably across various access patterns, whereas the advantages of complex algorithms in specific patterns are often accompanied by performance degradation in other patterns.
Fair evaluation of caching algorithms requires testing on a broad and representative set of workloads, with a well-tuned LRU serving as the strong baseline.
For teams building large model inference services, a well-implemented and well-tuned LRU is often already a quite powerful choice, and alternative solutions should be comprehensively tested on actual workloads before a decision is made.
LRU (Least Recently Used) is a cache eviction algorithm whose core idea is to evict the data that has not been accessed for the longest time.
LRU does not rely on predictions of future access patterns and incurs almost no additional computational overhead.
5 more timeline events
All Facts (16)
现代数据库如MySQL InnoDB和PostgreSQL内部都实现了缓冲池（Buffer Pool）机制，通过LRU等算法管理缓存替换策略
95%UnverifiedCache eviction policies include LRU, LFU, and weighted schemes that consider prefix length or recomputation cost; frequent cache eviction reduces the effectiveness of cache-aware routing.
50%UnverifiedGPU expert weight caching typically uses a replacement strategy similar to LRU (Least Recently Used), prioritizing retention of expert weights that have been frequently invoked recently.
50%UnverifiedFor teams building large model inference services, a well-implemented and well-tuned LRU is often already a quite powerful choice, and alternative solutions should be comprehensively tested on actual workloads before a decision is made.
50%UnverifiedAdvanced caching strategies require additional metadata tracking, scoring computation, and maintenance costs, and after accounting for these overheads, their net gain relative to LRU may shrink substantially.
50%UnverifiedLRU is extremely lightweight to implement and performs stably across various access patterns, whereas the advantages of complex algorithms in specific patterns are often accompanied by performance degradation in other patterns.
50%UnverifiedFair evaluation of caching algorithms requires testing on a broad and representative set of workloads, with a well-tuned LRU serving as the strong baseline.
50%UnverifiedLRU (Least Recently Used) is a cache eviction algorithm whose core idea is to evict the data that has not been accessed for the longest time.
50%UnverifiedLRU does not rely on predictions of future access patterns and incurs almost no additional computational overhead.
50%UnverifiedLRU is far harder to beat than KV-Cache papers suggest.
50%UnverifiedA large body of research has attempted to design smarter cache eviction strategies than LRU, such as deciding which cache entries to retain based on token importance, attention scores, or access frequency.
50%UnverifiedThe classic LRU algorithm is prone to cache pollution in one-time large-batch read scenarios such as full-table scans, evicting hot data from the cache.
50%UnverifiedUsing an LRU (Least Recently Used) caching strategy allows hot site data to be cached in memory while cold site data falls back to disk or a database.
50%UnverifiedThe LRU cache replacement policy requires at least 24 bytes of metadata overhead per entry on 64-bit systems (16 bytes for a doubly linked list + 8 bytes for a timestamp).
50%UnverifiedRedis的LRU实现并非教科书中严格的链表方案，而是采用近似LRU算法（随机采样+淘汰池）
50%UnverifiedOrvieto等人于2022年提出了Linear Recurrent Unit（LRU），使用对角复数矩阵作为循环核心
50%