Unverified50% confidenceSolutionExact time
GAD-RL weights the forward KL divergence using the student's predicted probability for the teacher's Top-1 token, suppressing local auxiliary updates when the student's support for the teacher's recommended token is low
1
Sources
50%
Confidence
Long-term
Relevance
10/1/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn traditional white-box distillation, the student model can directly access the complete probability distribution (logits) of the teacher model's output layer, whereas commercial large language model APIs typically return only the final generated text or top-k token probabilities.64% similarUnverifiedDirect Preference Optimization（DPO，2023年）通过数学变换将RLHF的两阶段问题压缩为单阶段监督学习，无需显式奖励模型64% similarUnverifiedRLCD constructs positive and negative sample pairs for contrastive learning, combines reinforcement learning signals to distill model output distributions, and has advantages in scenarios that do not require manually annotated rewards.64% similarVerifiedKnowledge distillation transfers the full probability distribution information output by the teacher model via soft labels, producing an implicit data augmentation effect.62% similarUnverifiedGNoME采用主动学习策略，迭代式地生成候选、预测稳定性、筛选最有价值结构送入DFT计算验证，再更新模型61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/963913API
curl https://kongchang.com/api/v1/knowledge/claims/963913MCP
get_claim(id=963913)