Unverified50% confidenceFactExact time
SDFT refers to Self-Distillation Fine-Tuning, that is, introducing a self-distillation objective during the fine-tuning stage, with the teacher and student evolving synchronously, which falls under the category of online self-distillation.
1
Sources
50%
Confidence
Long-term
Relevance
10/27/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedDistillation can be viewed as an implicit, data-constrained form of model merging—distilling a student using the averaged outputs of multiple teacher models is effectively performing a soft merge at the level of output distributions.63% similarUnverifiedIn on-policy distillation, the student model first generates trajectories itself, and then the teacher model provides supervision signals for these paths, thereby avoiding the distribution shift between offline data and student behavior.63% similarUnverifiedThe paper "Beyond the Answer: Student-Conditioned Trace Construction for Reasoning Distillation" proposes the ReDiST framework for constructing reasoning traces that improve learning effectiveness in reasoning distillation.62% similarVerifiedDistillation technology enables the student model to learn and reproduce the teacher's multi-step generation trajectory, compressing a generation process of hundreds of steps down to just a few.62% similarUnverifiedOutput distillation involves directly using text generated by a teacher model—such as dialogues and reasoning chains—as training data to fine-tune a student model, without requiring access to the teacher's internal parameters.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1101810API
curl https://kongchang.com/api/v1/knowledge/claims/1101810MCP
get_claim(id=1101810)