H2O
A KV cache eviction method based on token importance scoring. It identifies and retains important tokens by accumulating attention weights while discarding low-scoring tokens, reducing GPU memory consumption during inference.
Timeline (last 90 days)
Score-based eviction methods such as SnapKV and H2O use attention scores to determine which tokens are unimportant and delete them.
Common KV cache eviction methods include H2O, ScissorHands, and StreamingLLM, whose core logic is to assign an importance score to each token and remove low-scoring tokens.
The H2O eviction strategy dynamically eliminates unimportant historical tokens based on attention scores.
H2O is a dynamic sparse method based on Top-k selection, SnapKV is a method based on cumulative attention scores, and MInference is a structured sparse method with block sparsity.
All Facts (4)
Score-based eviction methods such as SnapKV and H2O use attention scores to determine which tokens are unimportant and delete them.
50%UnverifiedCommon KV cache eviction methods include H2O, ScissorHands, and StreamingLLM, whose core logic is to assign an importance score to each token and remove low-scoring tokens.
50%UnverifiedThe H2O eviction strategy dynamically eliminates unimportant historical tokens based on attention scores.
50%UnverifiedH2O is a dynamic sparse method based on Top-k selection, SnapKV is a method based on cumulative attention scores, and MInference is a structured sparse method with block sparsity.
50%