Unverified50% confidenceFactExact time
Reference Sliding Window Attention (R-SWA) introduces a fixed reference position on top of the sliding window, keeping the KV cache at a constant size during decoding while not sacrificing cross-page context consistency.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedFP8 KV cache is a quantization technique that compresses the Key-Value cache in the attention mechanism from FP16 or BF16 to 8 bits, which can halve the GPU memory usage of the KV cache70% similarVerifiedKV Cache avoids redundant computation by caching already-computed Key and Value matrices, and advanced techniques such as PagedAttention further optimize cache memory management efficiency.70% similarUnverifiedKV Cache memory usage is proportional to the number of layers × number of attention heads × context length × precision; GQA reduces cache size by having multiple query heads share the same set of Key-Value heads, and this design has been adopted by the Llama 2/3 and Qwen series.68% similarUnverifiedBaidu's UnlimitedOcr is a 3B-parameter OCR model with a context length of 32,768 tokens, built on DeepSeek OCR, replacing all decoder attention layers with Reference Sliding Window Attention (R-SWA), keeping the KV cache constant, and can transcribe dozens of pages in a single forward pass.68% similarVerifiedThe KV Cache (Key-Value Cache) strategy stores already-computed attention key-value pairs in GPU memory for reuse, which consumes a large amount of GPU memory resources.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1039070API
curl https://kongchang.com/api/v1/knowledge/claims/1039070MCP
get_claim(id=1039070)