Verified90% confidenceFactExact time
Google's Switch Transformer, released in 2021, applied MoE at scale to Transformer language models, expanding model capacity to the trillion-parameter level.
12
Sources
90%
Confidence
Long-term
Relevance
7/16/2026
First Seen
Sources
Related Claims
UnverifiedSwitch Transformer将专家数量扩展至2048个，2022年谷歌GLaM模型进一步验证了MoE在多任务泛化上的优势83% similarUnverifiedGshard（2020）首次实现了万亿参数MoE模型的分布式训练，Switch Transformer（2022）将每个token的路由选择收敛为单一专家70% similarUnverifiedAfter the Transformer architecture emerged in 2017, Google, OpenAI, Meta, and other organizations each completed their own preliminary research on large language models based on this architecture within roughly the same year.70% similarUnverifiedSwitch Transformer, proposed in 2021, simplified the number of experts routed to per token to 1.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/536153API
curl https://kongchang.com/api/v1/knowledge/claims/536153MCP
get_claim(id=536153)