Megatron
A large-scale language model training framework developed by NVIDIA. It supports multiple parallelism strategies, including tensor parallelism and pipeline parallelism, and is widely used for distributed pre-training and fine-tuning of very large-scale models.
Timeline (last 90 days)
Nvidia's frameworks such as NeMo and Megatron, as well as its own models (such as Nemotron), require massive amounts of corpus.
In RL training, the training side commonly uses the Megatron framework while the inference side relies on high-throughput engines such as vLLM; the accumulated differences between the two systems in computation order, numerical precision, and rounding handling can affect RL stability.
The RL-Kernel team ran a 200-step Qwen3-8B GRPO training task on 8 AMD MI300X cards, with results showing zero logprob mismatch between the Megatron training side and the vLLM rollout side.
The training framework may adopt a reduction strategy with cross-tensor-parallel sharding for throughput, while the inference framework uses a different kernel in KV-cache optimization mode; even with identical inputs, the last few ULPs of the output may differ.
All Facts (4)
Nvidia's frameworks such as NeMo and Megatron, as well as its own models (such as Nemotron), require massive amounts of corpus.
50%UnverifiedIn RL training, the training side commonly uses the Megatron framework while the inference side relies on high-throughput engines such as vLLM; the accumulated differences between the two systems in computation order, numerical precision, and rounding handling can affect RL stability.
50%UnverifiedThe RL-Kernel team ran a 200-step Qwen3-8B GRPO training task on 8 AMD MI300X cards, with results showing zero logprob mismatch between the Megatron training side and the vLLM rollout side.
50%UnverifiedThe training framework may adopt a reduction strategy with cross-tensor-parallel sharding for throughput, while the inference framework uses a different kernel in KV-cache optimization mode; even with identical inputs, the last few ULPs of the output may differ.
50%