Weight Reloading
In reinforcement learning post-training, the process of synchronizing updated model weights from the training engine to the inference engine. Because training and inference often use different parallelism strategies, weights must be resharded before the inference side can load them, making this a key factor in the efficiency of the RL training loop.
Timeline (last 90 days)
Completing weight refit with general-purpose collective communication primitives requires a two-step approach of 'first all-gather to aggregate into complete weights, then split according to the target layout', which incurs extra GPU memory usage for the complete weights and redundant communication in between.
In RL post-training, weight refit is not a one-time operation but is performed on every training iteration.
All Facts (2)
Completing weight refit with general-purpose collective communication primitives requires a two-step approach of 'first all-gather to aggregate into complete weights, then split according to the target layout', which incurs extra GPU memory usage for the complete weights and redundant communication in between.
50%UnverifiedIn RL post-training, weight refit is not a one-time operation but is performed on every training iteration.
50%