SkyRL
SkyRL is a reinforcement learning training infrastructure developed by Reflection. It supports LoRA, full fine-tuning and low-precision RL, and integrates vLLM with Megatron-Bridge.
Timeline (last 90 days)
SkyRL (developed by NovaSkyAI) provided Day-0 reinforcement learning training support for Reflection-Beam-1.0, covering LoRA fine-tuning, BF16 full-parameter fine-tuning and MXFP8-precision RL training, and uses vLLM for rollout sampling.
SkyRL's Day-0 support unifies LoRA, full BF16 fine-tuning, and MXFP8 reinforcement learning into vLLM inference and the Megatron-Bridge training framework.
SkyRL's distributed training solution R3 is already able to work in conjunction with Pipeline Parallelism.
During SkyRL's weight synchronization, the FP32 router weights are fully preserved to avoid numerical instability caused by low-precision conversion.
The SkyRL team announced Day-0 reinforcement learning support for Reflection's Beam model, meaning the Beam model can be plugged into the full RL training pipeline on the day of its release.
This LoRA decoding throughput improvement was achieved under a configuration of 32 concurrent requests and PP2×TP8, an increase of about 62%.
The 62% improvement is a result at the decoding stage, not an end-to-end training speedup ratio.
SkyRL's official report shows that by properly configuring expert adapters in vLLM, LoRA decoding throughput increased from 276 tok/s/GPU to 447 tok/s/GPU.
All Facts (8)
SkyRL (developed by NovaSkyAI) provided Day-0 reinforcement learning training support for Reflection-Beam-1.0, covering LoRA fine-tuning, BF16 full-parameter fine-tuning and MXFP8-precision RL training, and uses vLLM for rollout sampling.
50%UnverifiedSkyRL's Day-0 support unifies LoRA, full BF16 fine-tuning, and MXFP8 reinforcement learning into vLLM inference and the Megatron-Bridge training framework.
50%UnverifiedSkyRL's distributed training solution R3 is already able to work in conjunction with Pipeline Parallelism.
50%UnverifiedDuring SkyRL's weight synchronization, the FP32 router weights are fully preserved to avoid numerical instability caused by low-precision conversion.
50%UnverifiedThis LoRA decoding throughput improvement was achieved under a configuration of 32 concurrent requests and PP2×TP8, an increase of about 62%.
50%UnverifiedThe 62% improvement is a result at the decoding stage, not an end-to-end training speedup ratio.
50%UnverifiedSkyRL's official report shows that by properly configuring expert adapters in vLLM, LoRA decoding throughput increased from 276 tok/s/GPU to 447 tok/s/GPU.
50%UnverifiedThe SkyRL team announced Day-0 reinforcement learning support for Reflection's Beam model, meaning the Beam model can be plugged into the full RL training pipeline on the day of its release.
50%