Unverified50% confidenceFactExact time
In confidence-gated distillation, the distillation signal for a sample is only incorporated into training when the teacher model's prediction confidence for that sample exceeds a threshold.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedTeacher forcing introduces the exposure bias problem: during training the real target token is fed as input, while during inference the model's own predicted token is used, and prediction errors accumulate and amplify in subsequent steps.77% similarUnverified蒸馏训练范式导致开源模型学习到经过高度过滤、几乎不包含失败和纠错过程的数据分布，使模型形成不愿自我纠正的性格74% similarUnverified过拟合的统计原理是当模型参数量相对训练数据过多时，模型会记住噪声而非学到规律72% similarUnverified通过蒸馏获得的学生模型往往只继承教师模型的能力，而无法复制其经过RLHF等方法植入的安全对齐机制72% similarUnverifiedResearch directions for improving model calibration include introducing uncertainty tokens through supervised fine-tuning, performing post-calibration on probability distributions using temperature sampling, and training models to proactively output the ability to refuse to answer.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1003161API
curl https://kongchang.com/api/v1/knowledge/claims/1003161MCP
get_claim(id=1003161)