Concept
Routing Redundancy
A property of mixture-of-experts models whereby different experts are functionally substitutable, so that many tokens routed to suboptimal experts still produce little change in the output distribution; it is the precondition for maintaining quality after expert pruning without retraining.
Timeline (last 90 days)
Oct 16
MoE routing redundancy is usually high: different experts are functionally substitutable to a considerable degree, and for many tokens even reassigning them to suboptimal experts changes the output distribution little.
Unverified50%