#on-policy distillation
3 related articles
On-Policy Distillation Explained: Prin…
·3 min
On-Policy Distillation Explained: Principles, Advantages, and Applications — A Comprehensive Guide
A deep dive into On-Policy Distillation: core principles, key differences from Off-Policy methods, and applications in model compression, reasoning transfer, RLHF alignment, and self-improvement.
Read more →
·7 min
DeepSeek's Mysteriously Retracted Paper: Breaking AI's Multimodal Reasoning Bottleneck with Visual Primitives
DeepSeek's paper 'Thinking with Visual Primitives' was online for just 4 hours before being pulled. It uses bounding boxes and points as reasoning primitives, letting models 'point at' images to outperform GPT, Gemini, and Claude on maze navigation and counting.
Read more →
·4 min
DeepSeek-V4 Deep Dive: Million-Token Context Architecture Breakthrough and Core Innovations
Deep dive into DeepSeek-V4: 1.6T-parameter MoE, CSA+HCA hybrid attention, MHC & MUON optimizer. Inference FLOPs drop to 27% of V3.2, redefining open-source LLM SOTA.
Read more →