on-policy
A training sampling strategy in which training data is generated in real time by the current model, ensuring that the training distribution matches the inference distribution and avoiding distribution mismatch.
Timeline (last 90 days)
ORL adopts on-policy residual learning: training data is generated in real time by the current model to ensure that the training distribution matches the inference distribution.
PPO is an on-policy algorithm, requiring that the data used in each update must come from the current policy.
Mainstream RL algorithms such as PPO require that the policy used to generate samples be as consistent as possible with the policy currently being optimized (on-policy); otherwise, off-policy data introduces bias, reduces sample efficiency, and can potentially cause training instability.
All Facts (3)
ORL adopts on-policy residual learning: training data is generated in real time by the current model to ensure that the training distribution matches the inference distribution.
50%UnverifiedPPO is an on-policy algorithm, requiring that the data used in each update must come from the current policy.
50%UnverifiedMainstream RL algorithms such as PPO require that the policy used to generate samples be as consistent as possible with the policy currently being optimized (on-policy); otherwise, off-policy data introduces bias, reduces sample efficiency, and can potentially cause training instability.
50%