Unverified50% confidenceFactExact time
The Mistral model family uses Sliding Window Attention, with a window size of 4,096 tokens per layer.
1
Sources
50%
Confidence
Long-term
Relevance
8/31/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedSliding window attention allows each token to attend only to context within a fixed window, reducing complexity to O(n·w), where w is the window size.70% similarUnverifiedLaguna S 2.1 使用 12 层全局注意力 + 36 层滑动窗口注意力的混合方案，支持 1,048,576 Token（约 100 万）上下文窗口62% similarUnverifiedAt the time, even the larger model context windows were only on the order of tens of thousands of tokens.60% similarUnverified固定长度切片按token数截断，通常配合10到20%滑动窗口重叠，chunk size一般设置在256到512 token59% similarUnverifiedWhen the context window is extended from 128K to 1M, the attention computation theoretically increases by approximately 60 times.58% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/827311API
curl https://kongchang.com/api/v1/knowledge/claims/827311MCP
get_claim(id=827311)