Prime Mega MoE
A fused MoE kernel usable for training, built by extending the Mega MoE kernel on the basis of DeepSeek; it implements backpropagation support and improves MoE training efficiency.
Timeline (last 90 days)
Prime Intellect released Prime Mega MoE, adding backpropagation to DeepSeek's fused Mega MoE kernel, making it trainable.
Prime Mega MoE's forward pass speed reaches 3.11x (relative to the baseline implementation).
The 3.11x is kernel-level forward acceleration, while 34% is end-to-end training step speed improvement; the two are measured on different bases.
Prime Mega MoE is released as a standalone component, lowering the barrier for other frameworks to adopt a high-performance MoE kernel.
All Facts (4)
Prime Mega MoE's forward pass speed reaches 3.11x (relative to the baseline implementation).
50%UnverifiedThe 3.11x is kernel-level forward acceleration, while 34% is end-to-end training step speed improvement; the two are measured on different bases.
50%UnverifiedPrime Mega MoE is released as a standalone component, lowering the barrier for other frameworks to adopt a high-performance MoE kernel.
50%UnverifiedPrime Intellect released Prime Mega MoE, adding backpropagation to DeepSeek's fused Mega MoE kernel, making it trainable.
50%