Verified65% confidenceFactExact time
The Mixture of Experts (MoE) architecture was scaled up within the Transformer framework by Google Brain in 2017; its core idea is to group model parameters into multiple expert sub-networks, with only a small subset activated during each inference.
3
Sources
65%
Confidence
Long-term
Relevance
7/16/2026
First Seen
Sources
Related Claims
UnverifiedGoogle Brain于2017年在《Outrageously Large Neural Networks》论文中将MoE规模化应用于Transformer框架82% similarVerifiedThe revival of modern MoE can be traced back to the 2017 paper published by the Google Brain team, 'Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer,' which was the first to integrate sparse MoE layers into large sequence models.78% similarUnverifiedGoogle 的 Switch Transformer 和 Mixtral 采用了混合专家模型（MoE）思想，通过门控网络将输入路由到不同专家子网络并只激活部分参数78% similarUnverifiedThe MoE architecture divides the model into multiple expert sub-networks, activating only a small portion during each inference, reducing computational costs while maintaining knowledge capacity; Google's Gemini and Mistral's Mixtral series both adopt this architecture.76% similarVerifiedGoogle systematized the Mixture of Experts architecture design in its 2017 Sparsely-Gated MoE paper.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/536194API
curl https://kongchang.com/api/v1/knowledge/claims/536194MCP
get_claim(id=536194)