Unverified50% confidenceFactExact time
WISD adopts on-policy self-distillation, where the training signal comes from comparing parallel trajectories generated in real time by the original model and the weakened model on the same unlabeled prompt, with the comparison occurring in policy space rather than answer space.
1
Sources
50%
Confidence
Long-term
Relevance
10/30/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedWISD constructs weakened models through two approaches, low-bit quantization and magnitude pruning; in the method these two operations are used to generate training signals rather than for deployment optimization.79% similarUnverifiedJOLT uses on-policy training, where the training data is generated by the model's current policy rather than coming from a fixed offline dataset.77% similarUnverifiedModels are trained to make responses appear correct rather than to actually get the work done, and these two objectives are not naturally aligned in training signals.72% similarVerifiedThe train-serve consistency problem refers to logical discrepancies between offline feature computation during training and online real-time feature computation during serving, which can cause the model's online performance to be severely worse than its offline evaluation.72% similarUnverifiedThe training data for on-policy distillation is generated by real-time sampling of the student model under its current parameters, rather than being a pre-prepared fixed corpus, which contrasts with offline distillation.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1114236API
curl https://kongchang.com/api/v1/knowledge/claims/1114236MCP
get_claim(id=1114236)