Unverified50% confidenceFactExact time
Sliding window attention lets each token attend only to neighboring tokens within a fixed-length window, thereby reducing attention computation and the KV cache from linear growth with sequence length to constant level.
1
Sources
50%
Confidence
Long-term
Relevance
10/21/2026
First Seen
Sources
Related Entities
Related Claims
Unverified上下文越长，模型需要缓存的 KV Cache 越大，KV Cache 使模型生成新 Token 时无需重新计算所有历史 Token 的注意力权重79% similarUnverifiedOn-device deployment often comes with mechanisms such as KV cache quantization (e.g., compressing to 4 or 8 bits), grouped-query attention (GQA), and sliding-window or sparse attention.75% similarUnverifiedReference Sliding Window Attention (R-SWA) introduces a fixed reference position on top of the sliding window, keeping the KV cache at a constant size during decoding while not sacrificing cross-page context consistency.72% similarUnverifiedThe KV Cache size of sliding window attention is constant, achieving O(1) memory usage rather than O(n).71% similarUnverifiedThe KV cache stores key-value pairs from Transformer attention computations and grows linearly with sequence length.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1082479API
curl https://kongchang.com/api/v1/knowledge/claims/1082479MCP
get_claim(id=1082479)