Concept
reweighting bias
A statistical bias introduced by the random horizon: because tokens are supervised with different probabilities under different horizons, simple averaging causes the gradient estimate to deviate from the full-trajectory supervision objective.
Timeline (last 90 days)
Nov 17
Random horizons introduce a reweighting bias: tokens have different probabilities of being included in supervision under different horizons, and simple averaging causes the gradient estimate to deviate from the goal of full-trajectory supervision; the authors characterize this bias and derive the trade-off between compute and reweighting.
Unverified50%