Unverified50% confidenceReleaseExact time
Unsloth adds DeepSpeed ZeRO-2 and ZeRO-3 support, including 4-bit QLoRA, full fine-tuning, DPO and CPU offload; the DDP + gradient checkpointing LoRA training issue that previously caused crashes has been fixed.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/15/2026
First Seen
Valid until: 1/13/2027
Sources
Related Entities
Related Claims
UnverifiedZeRO was proposed by the DeepSpeed team in 2020 and eliminates redundancy in data parallelism through three stages: ZeRO-1 shards optimizer states, ZeRO-2 shards gradients, and ZeRO-3 shards model parameters.71% similarUnverifiedNVIDIA的Megatron-LM、Microsoft的DeepSpeed为大规模分布式训练提供基础设施，ZeRO用于降低GPU显存占用68% similarUnverified微软DeepSpeed提出的ZeRO技术分为三个阶段：Stage-1分片优化器状态可将内存需求降低约4倍，Stage-2额外分片梯度，Stage-3进一步分片模型参数66% similarUnverifiedThe RL-Kernel team ran a 200-step Qwen3-8B GRPO training task on 8 AMD MI300X cards, with results showing zero logprob mismatch between the Megatron training side and the vLLM rollout side.65% similarUnverifiedDeepSpeed's ZeRO optimizer can reduce the GPU memory required per device to 1/N of the original by sharding optimizer states, gradients, and model parameters.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1039702API
curl https://kongchang.com/api/v1/knowledge/claims/1039702MCP
get_claim(id=1039702)