Paper
Rethinking Routing Exploration in Modern Mixture-of-Experts
An academic paper studying the routing exploration problem in mixture-of-experts (MoE) models. It argues that the combination of fine-grained expert partitioning and shared experts leads to an exploration dilemma, and proposes the MirrorMoE method.
Timeline (last 90 days)
Oct 16
The paper "Rethinking Routing Exploration in Modern Mixture-of-Experts" points out that fine-grained expert partitioning combined with shared experts produces an exploration dilemma, weakening each other.
Unverified50%