Concept
random horizon
A training strategy in which a shared upper bound on response length (horizon) is randomly sampled for each training batch, and supervision is applied only to tokens within the horizon, thereby reducing redundant computation.
Timeline (last 90 days)
Nov 17
Stochastic Horizons On-Policy Distillation (SH-OPD) is an improved on-policy distillation method whose core idea is to randomly vary the horizon length of rollouts, reducing supervision of redundant positions while retaining access to deep response states.
Unverified50%