Unverified50% confidenceFactExact time
微软DeepSpeed提出的ZeRO技术分为三个阶段：Stage-1分片优化器状态可将内存需求降低约4倍，Stage-2额外分片梯度，Stage-3进一步分片模型参数
1
Sources
50%
Confidence
Long-term
Relevance
8/21/2026
First Seen
Sources
Related Claims
UnverifiedZeRO was proposed by the DeepSpeed team in 2020 and eliminates redundancy in data parallelism through three stages: ZeRO-1 shards optimizer states, ZeRO-2 shards gradients, and ZeRO-3 shards model parameters.79% similarUnverifiedDeepSpeed's ZeRO optimizer can reduce the GPU memory required per device to 1/N of the original by sharding optimizer states, gradients, and model parameters.73% similarUnverifiedNVIDIA的Megatron-LM、Microsoft的DeepSpeed为大规模分布式训练提供基础设施，ZeRO用于降低GPU显存占用61% similarUnverifiedLoRA（Low-Rank Adaptation）的核心思想是利用 SVD 的低秩近似原理，将大模型微调成本压缩至可在消费级 GPU 上运行的规模59% similarUnverified不考虑成本时，前端编程能力排名为：参照模型 > Kimi K3 > Grok 4.6 > DeepSeek V4 Pro > DeepSeek V4 Flash59% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/781272API
curl https://kongchang.com/api/v1/knowledge/claims/781272MCP
get_claim(id=781272)