Unverified50% confidenceFactExact time
Qwen3-Next and Kimi Linear both adopt a 3:1 ratio of linear attention to softmax attention.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/14/2026
First Seen
Valid until: 1/12/2027
Sources
Related Entities
Related Claims
UnverifiedQwen3 27B combines three linear attention layers with one full attention layer in a recurrent structure at a ratio of 3:1.78% similarUnverifiedKimi K3 adopts the Kimi Delta Attention hybrid linear attention mechanism, which reduces attention computation complexity from O(N²) to O(N) through kernel function approximation.72% similarUnverifiedK3 uses a mixture of linear attention layers and standard attention layers to seek a balance between accuracy and efficiency.71% similarUnverifiedKimi K3 employs Attention Vigilance technology, constructing independent long-fork pathways through all layers for the attention path to address the deep dilution problem.71% similarUnverifiedMiniMax-01 and Jamba push the ratio of linear attention to softmax attention to 7:1 or 8:1, and additionally stack an MoE structure on top.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1029831API
curl https://kongchang.com/api/v1/knowledge/claims/1029831MCP
get_claim(id=1029831)