Unverified50% confidenceFactExact time
The model's layer composition is 54 layers of KDA (Kimi Delta Attention) linear attention, 17 layers of BCSA compressed sparse attention, and 1 full attention layer (layer 72).
1
Sources
50%
Confidence
Long-term
Relevance
10/5/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedDense attention layers require a complete KV Cache, whereas linear layers rely on hidden states.75% similarUnverifiedCommon attention mechanism modules for model improvement include the SE attention mechanism, CBAM channel and spatial attention, and ECA efficient channel attention.74% similarUnverifiedIQuest-Q1 employs a mixed ratio of 3 layers of sliding window attention combined with 1 layer of full attention, with only 25 out of 88 layers expanding the KV cache as context grows.72% similarUnverifiedKimi K3 adopts the Kimi Delta Attention hybrid linear attention mechanism, which reduces attention computation complexity from O(N²) to O(N) through kernel function approximation.70% similarUnverifiedK3 uses a mixture of linear attention layers and standard attention layers to seek a balance between accuracy and efficiency.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/977110API
curl https://kongchang.com/api/v1/knowledge/claims/977110MCP
get_claim(id=977110)