Unverified50% confidenceFactExact time
The reverse KL divergence has a "mode-seeking" property: when the teacher distribution is multimodal, the student is pulled toward one of the teacher's dominant modes.
1
Sources
50%
Confidence
Long-term
Relevance
10/20/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedKL divergence measures the difference between two probability distributions and appears directly in the training objective of knowledge distillation, where the student model's output distribution is required to approximate the teacher's distribution in order to minimize the KL divergence between the two.72% similarUnverifiedThe researchers provide a theoretical analysis under an idealized bidirectional KL objective, proving that the expected reward of the optimal policy is no lower than the teacher's, while retaining a guaranteed proportion of the teacher's probability mass.68% similarUnverifiedThe teacher model in distillation does not have to be larger than the student model; it can be a domain expert model of comparable size.67% similarVerifiedKrashen's Input Hypothesis holds that learners need to be exposed to comprehensible input slightly above their current level (i+1).64% similarUnverifiedIn real classroom experiments, teachers need to act as hallucination filters, correcting occasional erroneous guidance from the AI.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1074091API
curl https://kongchang.com/api/v1/knowledge/claims/1074091MCP
get_claim(id=1074091)