ReDiST
A student-centered reasoning distillation framework that constructs reasoning trajectories suited to the target student's cognitive state by revising trajectories based on student drafts, using a reader model to filter and rank them, and balancing distillation loss against answer loss.
Timeline (last 90 days)
The paper "Beyond the Answer: Student-Conditioned Trace Construction for Reasoning Distillation" proposes the ReDiST framework for constructing reasoning traces that improve learning effectiveness in reasoning distillation.
The core insight of ReDiST is that the teaching quality of a reasoning trace depends not only on whether its content is correct but also on whether it matches the target student's cognitive state, so trace construction should be student-centered.
The ReDiST workflow consists of three steps: student-draft-based trace revision, student-matched filtering and ranking, and balanced distillation loss, all completed in the offline stage, with only the distilled student model used at inference time.
The trace revision step of ReDiST starts from the student model's own drafts: for drafts with correct answers, the provided evidence is used to revise them so that their reasoning process is more rigorous; for drafts with wrong answers, the verified answer is used to provide additional guidance for repair.
ReDiST uses a frozen reader model matched to the student to evaluate, based on the question, evidence, and candidate traces, their answer-recovery ability and verified-answer probability, ranking the original traces and the edited candidates.
The reader model plays the role of a "simulated student" in ReDiST; it is usually a frozen model of a scale or capability similar to the target student, does not participate in training, and is used only to evaluate candidate traces.
Experiments show that on question-answering benchmarks, ReDiST outperforms answer-only supervision, original traces, and the evaluated trace-selection and rewriting baselines on both the source task and cross-benchmark tests.
The paper's experiments found that the supervision method that performs best on the source benchmark is not necessarily likewise optimal on the transfer benchmark, so evaluating the teaching utility of reasoning traces needs to consider cross-distribution generalization.
Through student-conditioned construction and filtering, ReDiST offers a new approach to personalized reasoning distillation.
All Facts (9)
The paper "Beyond the Answer: Student-Conditioned Trace Construction for Reasoning Distillation" proposes the ReDiST framework for constructing reasoning traces that improve learning effectiveness in reasoning distillation.
50%UnverifiedThe core insight of ReDiST is that the teaching quality of a reasoning trace depends not only on whether its content is correct but also on whether it matches the target student's cognitive state, so trace construction should be student-centered.
50%UnverifiedThe ReDiST workflow consists of three steps: student-draft-based trace revision, student-matched filtering and ranking, and balanced distillation loss, all completed in the offline stage, with only the distilled student model used at inference time.
50%UnverifiedThe trace revision step of ReDiST starts from the student model's own drafts: for drafts with correct answers, the provided evidence is used to revise them so that their reasoning process is more rigorous; for drafts with wrong answers, the verified answer is used to provide additional guidance for repair.
50%UnverifiedReDiST uses a frozen reader model matched to the student to evaluate, based on the question, evidence, and candidate traces, their answer-recovery ability and verified-answer probability, ranking the original traces and the edited candidates.
50%UnverifiedThe reader model plays the role of a "simulated student" in ReDiST; it is usually a frozen model of a scale or capability similar to the target student, does not participate in training, and is used only to evaluate candidate traces.
50%UnverifiedExperiments show that on question-answering benchmarks, ReDiST outperforms answer-only supervision, original traces, and the evaluated trace-selection and rewriting baselines on both the source task and cross-benchmark tests.
50%UnverifiedThe paper's experiments found that the supervision method that performs best on the source benchmark is not necessarily likewise optimal on the transfer benchmark, so evaluating the teaching utility of reasoning traces needs to consider cross-distribution generalization.
50%UnverifiedThrough student-conditioned construction and filtering, ReDiST offers a new approach to personalized reasoning distillation.
50%