Unverified50% confidenceFactExact time
Delta-rule linear attention is the underlying mechanism of language models such as Kimi Linear and Qwen3-Next.
1
Sources
50%
Confidence
Long-term
Relevance
11/3/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedKDA (Kernelized Delta Attention) kernels the Delta rule, attempting to find a smoother interpolation between the recursive state and attention.62% similarUnverifiedMost SSM and linear attention models use a linear attention mechanism at the bottom layer, with Mamba-2 implemented through the SSD structure and xLSTM implemented through covariance updates.62% similarUnverifiedMamba-2, Gated DeltaNet and KDA are three new state update mechanisms used to replace some attention layers, achieving efficient state memory through a recursive structure with linear complexity.62% similarUnverifiedHybrid language models alternate or combine full-attention layers with efficient layers such as linear attention and state space models (e.g., Mamba); full-attention layers must store key-value vectors for every historical token, with overhead growing linearly with sequence length, while linear attention layers replace the KV cache with a fixed-size recurrent state.61% similarVerifiedQwen3 introduces a dynamic switching mechanism between thinking mode and non-thinking mode.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1129578API
curl https://kongchang.com/api/v1/knowledge/claims/1129578MCP
get_claim(id=1129578)