Reverse KL
A directional form of KL divergence that tends to concentrate the student policy on high-probability regions of the teacher policy. It learns precisely but may omit some modes, and is commonly used to provide precise teacher corrections at critical moments.
Timeline (last 90 days)
Reverse KL favors refining the token ranking the student is already active on (mode-seeking), while forward KL is responsible for exploring candidates that the teacher's support covers but the student does not sufficiently attend to (mass-covering).
Using either KL divergence (forward or reverse) alone biases distillation toward a certain behavior pattern, either overly conservative or introducing too much noise.
All Facts (2)
Reverse KL favors refining the token ranking the student is already active on (mode-seeking), while forward KL is responsible for exploring candidates that the teacher's support covers but the student does not sufficiently attend to (mass-covering).
50%UnverifiedUsing either KL divergence (forward or reverse) alone biases distillation toward a certain behavior pattern, either overly conservative or introducing too much noise.
50%