Unverified50% confidenceFactExact time
Minimax has trained a 456B-parameter MoE using a linear attention approach, but still requires occasional use of full attention.
1
Sources
50%
Confidence
Long-term
Relevance
9/23/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedMiniMind covers the complete pipeline including Tokenizer training, pre-training, supervised fine-tuning (SFT), LoRA fine-tuning, DPO reinforcement learning, and model distillation.68% similarUnverifiedALiBi在注意力分数中加入与距离成线性比例的负偏置项，实现近程优先的注意力模式，且外推长度几乎不受训练长度限制65% similarUnverified目前发布的 H3 权重并未启用模型训练时使用的更快稀疏注意力路径，团队将发布一个刻意保守的稀疏注意力版本64% similarUnverified少样本示例（Few-shot Prompting）的工作原理基于Transformer注意力机制，无需更新模型权重，被称为上下文学习（In-context Learning）62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/942440API
curl https://kongchang.com/api/v1/knowledge/claims/942440MCP
get_claim(id=942440)