Unverified50% confidenceFactExact time
GShard于2020年首次在Transformer中实现了600B参数的MoE训练
1
Sources
50%
Confidence
Long-term
Relevance
8/18/2026
First Seen
Sources
Related Claims
UnverifiedFP8精度（Hopper架构H100引入）可将Transformer训练吞吐提升近2倍，若训练超大模型且框架支持Transformer Engine，H100相比A100的实际训练加速远超纸面算力差距62% similarUnverifiedGoogle 2017年提出的Sparsely-Gated MoE首次将稀疏门控引入Transformer规模的序列模型，可在不显著增加计算量前提下将模型容量扩大1000倍以上59% similarUnverifiedH100的Hopper架构引入了专门的Transformer Engine，支持FP8精度计算59% similarUnverifiedBlackwell的第二代Transformer Engine新增FP4精度支持，理论上可将Transformer模型的训练吞吐量相比H100提升2-4倍58% similarVerifiedTransformers库由Hugging Face维护，提供了数千个预训练模型的统一接口58% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/768300API
curl https://kongchang.com/api/v1/knowledge/claims/768300MCP
get_claim(id=768300)