Extrapolation
In knowledge distillation, a technique that constructs a stronger learning target by extending the log-probability shift of the teacher model relative to a reference policy, thereby strengthening the learning signal.
Core Facts
Timeline (last 90 days)
For incorrect rollouts, APSD adopts an extrapolation strategy, extending the target distribution beyond the self-teacher to strengthen correction.
The extrapolation target takes the form p_S + β(p_T - p_S) (β>1), may introduce larger variance, and needs to be used with clipping or KL constraints.
True scientific discovery typically requires extrapolation ability—jumping beyond the known data distribution to propose unprecedented hypotheses—which is currently a weakness of AI.
All Facts (3)
True scientific discovery typically requires extrapolation ability—jumping beyond the known data distribution to propose unprecedented hypotheses—which is currently a weakness of AI.
65%UnverifiedFor incorrect rollouts, APSD adopts an extrapolation strategy, extending the target distribution beyond the self-teacher to strengthen correction.
50%UnverifiedThe extrapolation target takes the form p_S + β(p_T - p_S) (β>1), may introduce larger variance, and needs to be used with clipping or KL constraints.
50%