MoE Routing Strategies Explained: From Top-K to Dynamic Routing

Mixture-of-Experts (MoE) models have become the mainstream path for scaling large models, and the routing strategy determines which experts each token is assigned to. A guide published by MarkTechPost systematically reviews 7 MoE routing methods, from the top-2 softmax gating used by Mixtral to the hash routing adopted by DeepSeek-V4, and notes the representative models for each. Based on that guide, this article breaks down the design logic and trade-offs of these strategies.

Why Routing Strategy Is the Core of MoE

The basic idea of MoE is to split a dense feed-forward network into multiple experts, activating only a small subset for each token, thereby controlling the computation per inference while total parameter count grows. The routing strategy answers a seemingly simple question: who should this token go to?

This decision directly affects three things. The first is load balancing—if routing always favors a few experts, the rest are effectively idle and the scaling gains are greatly diminished. The second is training stability—routing is a discrete choice, gradients cannot be directly backpropagated, and tricks are needed to make the gating network trainable. The third is inference efficiency—routing results determine the communication patterns and memory access patterns during expert parallelism. For this reason, differences in routing across models often reflect how they prioritize these objectives.

Gating network is the execution body of the routing strategy, typically a lightweight linear layer or small MLP. It takes the token's hidden representation, outputs a score or probability distribution for each expert, and decides allocation accordingly. The gating network is trained end-to-end together with the expert networks, but its parameter count is far smaller than the sum of the experts, so the computation overhead of routing itself is usually negligible. The output form of the gate (softmax, sigmoid, or an indicator function that directly takes top-K) determines how sparsity is produced and whether gradients can flow back smoothly. Understanding the structure of the gating network is the foundation for understanding the various routing variants that follow.

Expert parallelism is a distributed strategy used during MoE inference and training. Since the number of experts can reach dozens or even hundreds, a single device cannot hold all expert parameters, so different experts need to be placed on different GPUs. When a token is routed to an expert on a remote device, its hidden state must be transmitted across devices, forming all-to-all communication. The more imbalanced the routing results, the more concentrated the communication load, and some devices may become bottlenecks. Therefore, the routing strategy not only affects computation allocation but also directly shapes the communication patterns of the distributed system—an engineering dimension that cannot be ignored when evaluating routing schemes.

Top-K Routing: The Most Mainstream and Direct Approach

Top-2 Softmax: Mixtral's Classic Choice

The most widely known routing method is top-K gating. The gating network outputs a score for each expert for each token, takes the K experts with the highest scores, normalizes with softmax, and computes a weighted sum. Mixtral uses top-2 softmax: each token activates two experts.

This scheme is simple to implement and stable to train, and K=2 strikes a good balance between computation overhead and model capacity. Its weakness is that balance depends on auxiliary loss functions—without an additional load-balancing constraint, the gating network tends to collapse onto a few experts, forming a "winner-takes-all" situation.

Auxiliary Loss and Capacity Factor

The engineering details around top-K mainly focus on two points: the load-balancing auxiliary loss and the expert capacity factor. The former adds a penalty term to the training objective, encouraging tokens to spread evenly across experts; the latter sets an upper limit on the number of tokens each expert can process, with excess tokens dropped or passed through a residual path. A small capacity factor controls computation but brings token dropping; a large one wastes compute. The choice of these hyperparameters is an unavoidable trade-off when putting a routing strategy into practice.

Winner-takes-all is the typical failure mode of top-K gating in the absence of constraints. Because the gating network tends to give higher scores to experts that already perform better, experts that happen to receive more tokens early in training further reinforce their advantage, and eventually a few experts monopolize almost all tokens while the rest receive no effective gradients and degrade. Auxiliary loss breaks this positive feedback by penalizing the variance of expert load or encouraging uniform routing probabilities, but the penalty coefficient must be carefully tuned: too strong and it interferes with main-task learning, too weak and it cannot prevent collapse. This is also why the top-K scheme looks simple but is actually quite sensitive to tuning.

Token dropping means that when the number of tokens received by an expert exceeds the upper limit set by the capacity factor, the excess tokens are no longer computed by that expert but bypass it directly through the residual connection. Although dropping controls the computation of a single expert, the dropped tokens actually receive no expert processing at that layer, effectively losing model capacity. Excessive dropping during training weakens gradient signals, and during inference it may cause fluctuations in output quality. The capacity factor therefore becomes a tuning knob between computational efficiency and model quality, usually set empirically based on batch size and number of experts.

Expert-Choice Routing: Reversing the Allocation Direction

Expert-Choice routing flips the direction of the allocation problem. Traditional top-K has each token choose experts, whereas Expert-Choice lets each expert choose the tokens it most wants, with each expert receiving a fixed number of tokens within its capacity.

The value of this reversal is that load balancing becomes an intrinsic property of the routing mechanism rather than something corrected after the fact by auxiliary loss. Since each expert's load is hard-constrained, no expert sits idle or is overloaded during training. The cost is that some tokens may not be selected by any expert and require extra handling; the transpose of the routing matrix also changes implementation details. For scenarios that prioritize training stability and expert utilization, this is an attractive alternative.

Routing matrix transpose is key to understanding the implementation differences of Expert-Choice. Traditional top-K routing produces a token-to-expert assignment matrix of shape number of tokens times number of experts, with K non-zero entries retained per row; Expert-Choice first computes each expert's affinity for all tokens, then takes top-C per column (C being expert capacity), equivalent to transposing the original matrix and selecting by row. This transpose changes the dimensions of parallel computation: computation originally grouped by token must be regrouped by expert, imposing different requirements on batch processing implementations and communication primitives. In addition, tokens not selected by any expert must be marked and handled separately, usually passed directly through the residual or backed by a shared expert, which introduces additional masking logic.

Sigmoid Gating and Dynamic Routing

The guide also covers two lines of thought: sigmoid gating and dynamic routing. Compared with softmax, sigmoid gating scores each expert independently without normalization competition, thus allowing a token to match multiple experts at once and also allowing all experts to score low—this provides more flexible room for controlling sparsity.

Dynamic routing lets the number of experts activated per token vary with the input rather than being fixed at K. A simple input may need only one expert, while a complex input can call more. This on-demand allocation is theoretically closer to the original intent of MoE, but in practice it adds the problem of inconsistent computation graphs within a batch and places higher demands on the scheduling capabilities of the inference system.

Hash Routing: DeepSeek-V4's Deterministic Path

Another representative scheme mentioned in the guide is the hash routing adopted by DeepSeek^Ad-V4. Hash routing uses a deterministic function rather than a learnable gate to decide token assignment, eliminating the training and inference overhead of the gating network and naturally avoiding load collapse.

The trade-off of this design is clear: it exchanges determinism for simplicity and stability, but sacrifices the ability to intelligently assign experts based on content. It is better suited to scenarios that are sensitive to routing overhead or that want routing behavior to be fully predictable. Listing it alongside other methods also shows that MoE routing does not have only one evolutionary path of "smarter gating."

How to Choose a Routing Strategy

Looking at the 7 methods together, the selection logic can be reduced to a few dimensions: whether strong load balancing is needed, whether token dropping is acceptable, the budget for training and inference overhead, and whether routing behavior must be predictable.

Top-K softmax is a default option validated at scale and suits most teams building MoE from scratch; Expert-Choice has an advantage in scenarios with high load-balancing requirements; sigmoid gating offers more flexible sparsity control; dynamic routing suits teams willing to invest in systems engineering; hash routing is a pragmatic choice when determinism and low overhead are priorities. The value of the guide is not that it gives a single answer, but that it lays out these trade-offs so each team can make judgments based on its own constraints.

One-line Summary

A systematic review of the design logic and trade-offs of 7 MoE routing strategies, from top-2 softmax to hash routing.

Paragraph Summary

Based on a guide from MarkTechPost, this article breaks down 7 routing strategies for Mixture-of-Experts (MoE) models. It points out that routing strategy directly affects load balancing, training stability, and inference efficiency, and is a core design decision for MoE. Top-K softmax (such as Mixtral's top-2) is the most mainstream scheme, simple to implement but reliant on auxiliary loss and capacity factor to prevent expert collapse and token dropping; Expert-Choice reverses the allocation direction, letting experts choose tokens so that load balancing becomes an intrinsic property of the mechanism; sigmoid gating does not use normalized competition and provides more flexible sparsity control; dynamic routing lets the number of activated experts vary with input; DeepSeek-V4's hash routing replaces learnable gating with a deterministic function in exchange for simplicity, stability, and low overhead. The article ends with a framework for choosing based on load-balancing needs, tolerance for token dropping, overhead budget, and predictability.