Conceptrouter
Router Network
The gating component of an MoE model, responsible for selecting the experts activated for each token based on its representation; its decision is essentially a classification problem over token representations.
The gating component of an MoE model, responsible for selecting the experts activated for each token based on its representation; its decision is essentially a classification problem over token representations.