Unverified50% confidenceFactExact time
WISD's weakened model shares the same origin as the original model, sharing pretrained knowledge, tokenization, and reasoning style, thereby eliminating the teacher-student distribution mismatch problem.
1
Sources
50%
Confidence
Long-term
Relevance
10/30/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedWhen general reasoning paradigms fail in a specific domain, constraining the reasoning process with domain structure and then distilling reasoning trajectories with a teacher model may be an effective path for small models to acquire specialized capabilities.75% similarUnverifiedIn knowledge distillation, off-policy distillation uses trajectories generated by the teacher model as training samples, and suffers from the distribution shift problem where the state distribution visited by the teacher is inconsistent with the student's own state distribution.75% similarUnverified知识蒸馏的中间层对齐策略（如PKD、TinyBERT）通过强制学生模型的隐藏表示模仿教师模型的分布间接约束嵌入几何结构74% similarUnverifiedGenerating training data with a teacher model silently transfers the teacher's blind spots, stylistic habits and failure modes, and the student model inherits the specific shape of the teacher's mistakes.73% similarUnverifiedTraditional reasoning distillation provides the teacher model's complete reasoning trace together with the correct answer to the student model, but a reasoning trace, even when accompanied by the correct answer, does not necessarily provide effective supervision signals for a particular student.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1114238API
curl https://kongchang.com/api/v1/knowledge/claims/1114238MCP
get_claim(id=1114238)