Paper

ScissorHands

A KV cache token eviction method that selectively retains tokens based on importance scores derived from attention weights, reducing GPU memory consumption during long-context inference.

Timeline (last 90 days)

Oct 7

Common KV cache eviction methods include H2O, ScissorHands, and StreamingLLM, whose core logic is to assign an importance score to each token and remove low-scoring tokens.

Unverified50%

All Facts (1)

Unverified

Common KV cache eviction methods include H2O, ScissorHands, and StreamingLLM, whose core logic is to assign an importance score to each token and remove low-scoring tokens.

50%

Source Articles

AttSVD: Attention-Guided SVD Compresses KV Cache to Half the Memory Without Accuracy LossOct 7