Model
MiniMax-01
A hybrid-architecture model released by MiniMax that combines linear attention and full attention at a ratio of roughly 7:1 or 8:1. It also employs a Mixture-of-Experts (MoE) structure to improve parameter efficiency, allowing the model to scale capacity without proportionally increasing compute cost.
Timeline (last 90 days)
Oct 14
MiniMax-01 and Jamba push the ratio of linear attention to softmax attention to 7:1 or 8:1, and additionally stack an MoE structure on top.
Unverified50%