#MoE
28 related articles
Meta's 816B Model Activates Only 5% of Parameters: An Animated Deep Dive into MoE Architecture
Meta's latest 816B-parameter model activates only about 5% of parameters per token. This article uses a Reddit animation to explain MoE architecture's sparse activation, routing, and load balancing, and analyzes its inference cost advantages and deployment challenges.
iPhone Runs 35B MoE at 20.5 tok/s: The Scheduling Magic That Skips No Experts
OpenBMB's MiniCPM-V 4.7 uses Lookahead Expert Scheduling to run a 35B-A3B MoE model on iPhone at 20.5 tok/s without skipping experts, losing only 0.16 points accuracy. Open-sourced in llama.cpp-omni.
MiniMax M3.1 Flash Released: A 428B MoE Agent Model with Input at Just $0.15/M
MiniMax releases M3.1 Flash: a 428B-parameter MoE model with ~23B active parameters, 1M-token context, API input priced at $0.15 per million tokens, and weights planned for October 23. This article analyzes its sparse architecture, agent positioning, and pricing strategy.
llama.cpp MoE Cache Benchmark: Doubling 30B Model Decode Speed on a T4
llama.cpp's new MoE expert cache doubles Qwen3-30B-A3B decode speed on a free Tesla T4, from 22.6 to 43.2 tokens/s. Cache mechanics, config script, benchmarks, and concurrency pitfalls.
Running a 125B-Parameter LLM on a 12GB GPU: Strata Local Deployment Hands-On
Bilibili creator FT tests open-source Strata: run a 125B-parameter LLM locally on a 12GB-VRAM gaming PC. RTX 5070 hits 94 tokens/s, uses 11.2GB VRAM, installs via double-click script, OpenAI-compatible API.
DeepSeek Flash 1017: An LLM That Runs a Million Tokens on a Single GPU
DeepSeek v4.1-Flash 1017 uses a 284B sparse MoE, 13B activated per token, and HSQ-A compression to cut KV-cache by 85% and enable million-token context on a single A100.
Open Weights Approach the Frontier: 501B Beam and 816B Meta MoE Debut in the Same Week
Gemini 4 Argon goes GA, while Meta's 816B Muse Spark Open and Reflection's 501B Beam ship as open weights. We break down capabilities, pricing, and hardware thresholds.
StepFun Open-Sources Step 5 Preview: A 600B MoE, 1M-Context Agent Foundation
StepFun open-sources Step 5 Preview, a 600B sparse MoE model with 27B activated parameters and up to 1M token context. FP8 and BF16 weights are on Hugging Face and ModelScope, built for long-horizon agents.
Run a 125-Billion-Parameter LLM on a Home PC? Inside the 25k-Star Strata Local Deployment
GitHub's 25k-star Strata project makes running a 125-billion-parameter LLM locally on a home PC possible. Built on MoE architecture with VRAM/RAM tiering, OpenAI API compatible, deployable with 12GB VRAM and 32GB RAM—data never leaves your machine.
Fine-Grained Experts + Shared Experts: The Dilemma of Routing Exploration in MoE
Fine-grained and shared experts create a routing exploration dilemma: shared experts reduce token representation diversity and weaken exploration, preventing fine-grained expressiveness from being realized. MirrorMoE resolves this by shrinking the exploration space.
Meta Open-Sources an 816B Beast: The Opportunities and Barriers of Muse Spark Open
Meta releases Muse Spark Open, an open-weights model with 816B total parameters, 42B activated, a 1M-token context, and multimodality. This article breaks down its MoE architecture, download channels, hardware barriers, and Muse Model License restrictions.
Fish Audio Releases Drama 3: A 35B Open-Weights MoE Text-to-Speech Model
Fish Audio releases Drama 3, a 35B MoE text-to-speech model with ~3B active parameters, 83 languages, open weights under a revenue-capped community license, and natural language voice control.
AntAngelMed: A 103B Total / 6B Active Medical MoE Model
AntAngelMed is a medical MoE open-source model jointly developed by Ant Health and others, with 103B total parameters and only 6.1B activated, ranking first on MedBench v4, supporting 128K context, Apache-2.0 licensed, and efficiently deployable on your own infrastructure.
Ai2 Olmo-core 3: The Secret Behind 2.7x MoE Training Throughput
Ai2 open-sources Olmo-core 3, replacing FSDP with DDP to eliminate the MoE capacity tax. Expert pool grows 8→128 with only 5% throughput loss; 47B model hits 2.7x the old implementation.
Running MoE Models Beyond RAM: Expert Offloading and SSD Streaming in llama.cpp
A practical guide to running MoE large models beyond RAM with llama.cpp: using mmap, --n-cpu-moe expert offloading, and SSD streaming to run 100B+ parameter models on limited hardware, including memory estimation scripts, configuration parameters, and bottleneck diagnosis methods.
Reflection Beam: 501B Stored, 23B Awake — A Must-Read for Self-Hosters
Reflection AI releases Beam, a 501B-parameter open-weight MoE model with Apache 2.0 license and 1M context. We break down what 501B storage vs 23B activation means for self-hosters, its benchmarks, and engineering details.
AMD MoRI + vLLM: Wide Expert Parallelism and RDMA KV Cache Transfer for MoE
AMD engineers will demonstrate MoRI + vLLM for MoE serving at PyTorch Conference 2026, featuring Wide Expert Parallelism and RDMA KV cache transfer on AMD hardware.
DeepEP Ascend Deep Dive: DeepSeek's MoE Communication Library Ported to Huawei Ascend NPU
DeepSeek's MoE communication library DeepEP is now ported to Huawei Ascend NPU. We break down Dispatch, Combine, FP8 compression, compute-communication overlap, and benchmark bandwidth from EP8 to EP128.
Breaking Down DeepEP and DeepGEMM: MoE Communication and Matrix Computation Acceleration in Practice
Deep dive into DeepSeek's DeepEP and DeepGEMM Ascend: DeepEP achieves 90%+ theoretical bandwidth for MoE communication; MegaMoE fusion operator hits 852 TFLOPS, boosting training 1.5× and decode 2.5×.
How MoE Architecture Cuts Costs and Boosts Efficiency: Efficient Training of Biological Foundation Models
NVIDIA explores using Mixture of Experts (MoE) architecture to efficiently train biological foundation models, decoupling parameter scale from compute costs for DNA and protein modeling.