Unverified50% confidenceTradeoffExact time
MoE的All-to-All通信开销可能占据总训练时间的30%-50%，成为规模扩展的核心瓶颈
1
Sources
50%
Confidence
Long-term
Relevance
7/24/2026
First Seen
Sources
Related Claims
UnverifiedIn the training of hundred-billion-parameter large models, inter-node communication overhead can account for 30%–50% of total training time.80% similarUnverified在大模型训练中，通信开销往往占到总训练时间的30%以上75% similarUnverifiedAt the thousand-GPU scale, the latency of a single AllReduce operation can account for more than 30% of the per-step training time.63% similarUnverified大型训练任务中因故障和恢复造成的算力浪费可达总训练时间的5%到20%63% similarUnverifiedIn a typical ResNet-50, replacing all pooling and strided downsampling operations with spectral pooling could increase overall training time by 30%–50%.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/605003API
curl https://kongchang.com/api/v1/knowledge/claims/605003MCP
get_claim(id=605003)