on-policy rollout
In reinforcement learning, the process of sampling and generating sequences under the current policy distribution, used in algorithms such as GRPO to compute relative advantages within a group.
Timeline (last 90 days)
In on-policy rollouts, incorrect pseudo-label words rarely appear, so the group-relative advantage always aligns with clean transcriptions; this is the mechanistic explanation for GRPO's noise robustness.
On-policy rollout means sampling generated sequences under the distribution of the current policy (i.e., the student model itself), rather than using offline data produced by a fixed dataset or an old policy.
All Facts (2)
In on-policy rollouts, incorrect pseudo-label words rarely appear, so the group-relative advantage always aligns with clean transcriptions; this is the mechanistic explanation for GRPO's noise robustness.
50%UnverifiedOn-policy rollout means sampling generated sequences under the distribution of the current policy (i.e., the student model itself), rather than using offline data produced by a fixed dataset or an old policy.
50%