Kimi Linear
A hybrid architecture model released by Moonshot AI, adopting a hybrid design combining linear attention and softmax attention in a 3:1 ratio.
Timeline (last 90 days)
Qwen3-Next and Kimi Linear both adopt a 3:1 ratio of linear attention to softmax attention.
Kimi Linear, introduced by Moonshot AI, is a hybrid linear attention architecture whose core is Kimi Delta Attention (KDA); most layers use KDA, with a full attention block reusing DeepSeek-V3 MLA retained every four layers.
In a Kimi Linear, 8-card B200, concurrency-8 test, cached-prefix throughput improved by approximately 10.9x.
All Facts (3)
Qwen3-Next and Kimi Linear both adopt a 3:1 ratio of linear attention to softmax attention.
50%UnverifiedKimi Linear, introduced by Moonshot AI, is a hybrid linear attention architecture whose core is Kimi Delta Attention (KDA); most layers use KDA, with a full attention block reusing DeepSeek-V3 MLA retained every four layers.
50%UnverifiedIn a Kimi Linear, 8-card B200, concurrency-8 test, cached-prefix throughput improved by approximately 10.9x.
50%