#multi-teacher distillation
3 related articles
·4 min
Kimi K2/K3 Technical Speculation: Deep Dive into RL Hyperparameters and Multi-Teacher Distillation
Deep analysis of RL hyperparameter tuning challenges and 9-policy multi-teacher distillation in Kimi K2/K3 training, exploring the shift from scale to training craft.
Read more →
On-Policy Distillation Explained: Prin…
·3 min
On-Policy Distillation Explained: Principles, Advantages, and Applications — A Comprehensive Guide
A deep dive into On-Policy Distillation: core principles, key differences from Off-Policy methods, and applications in model compression, reasoning transfer, RLHF alignment, and self-improvement.
Read more →
·3 min
Claude Opus 4.8 Identifies Itself as DeepSeek: Data Contamination or Distillation? A Technical Analysis
Anthropic's Claude Opus 4.8 failed within 2 hours of launch, identifying itself as DeepSeek and Tongyi Qianwen in Chinese. Deep analysis of data contamination vs distillation hypotheses and multilingual alignment gaps.
Read more →