OPSD

A knowledge distillation variant that does not rely on a fixed offline teacher model; instead, the current policy or an enhanced version of it generates token-level supervision signals in real time during training and feeds them back to the model, which can alleviate the credit assignment problem under sparse rewards.