Unverified50% confidenceFactExact time
Google 的 Switch Transformer 和 Mixtral 采用了混合专家模型（MoE）思想，通过门控网络将输入路由到不同专家子网络并只激活部分参数
1
Sources
50%
Confidence
Long-term
Relevance
8/18/2026
First Seen
Sources
Related Claims
VerifiedThe Mixture of Experts (MoE) architecture was scaled up within the Transformer framework by Google Brain in 2017; its core idea is to group model parameters into multiple expert sub-networks, with only a small subset activated during each inference.78% similarVerifiedThe core idea of MoE (Mixture of Experts) can be traced back to the early mixture of experts framework proposed by Jacobs et al. in 1991, and Google's Switch Transformer in 2021 brought it into the era of large-scale language models.74% similarVerifiedAfter the successful deployment of MoE models represented by Google Switch Transformer and Mixtral, the MoE architecture has become the mainstream technical choice for ultra-large-scale language models.73% similarVerifiedGoogle's Switch Transformer, released in 2021, alleviates load imbalance by simplifying routing to a Top-1 selection and introducing an expert capacity mechanism.72% similarVerifiedGoogle validated the Mixture of Experts (MoE) architecture at scale in Switch Transformer.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/766339API
curl https://kongchang.com/api/v1/knowledge/claims/766339MCP
get_claim(id=766339)