Unverified50% confidenceFactExact time
2022年Google的《Switch Transformers》将MoE中选Top-K专家简化为选Top-1专家，并通过辅助负载均衡损失函数解决专家坍塌问题
1
Sources
50%
Confidence
Long-term
Relevance
7/21/2026
First Seen
Sources
Related Claims
VerifiedGoogle's Switch Transformer, released in 2021, alleviates load imbalance by simplifying routing to a Top-1 selection and introducing an expert capacity mechanism.80% similarVerifiedGoogle validated the Mixture of Experts (MoE) architecture at scale in Switch Transformer.80% similarUnverifiedMoE混合专家架构最早可追溯至1991年的学术论文，谷歌于2021年发布Switch Transformer将专家数量扩展至2048个71% similarVerifiedThe Mixture of Experts (MoE) architecture was scaled up within the Transformer framework by Google Brain in 2017; its core idea is to group model parameters into multiple expert sub-networks, with only a small subset activated during each inference.67% similarUnverifiedGoogle 的 Switch Transformer 和 Mixtral 采用了混合专家模型（MoE）思想，通过门控网络将输入路由到不同专家子网络并只激活部分参数66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/584867API
curl https://kongchang.com/api/v1/knowledge/claims/584867MCP
get_claim(id=584867)