Forward KL
A directional form of KL divergence that tends to make the student policy cover all modes of the teacher policy. It learns comprehensively but may lack precision, and is commonly used to maintain global coverage of teacher behavior.
Timeline (last 90 days)
Reverse KL favors refining the token ranking the student is already active on (mode-seeking), while forward KL is responsible for exploring candidates that the teacher's support covers but the student does not sufficiently attend to (mass-covering).
Using either KL divergence (forward or reverse) alone biases distillation toward a certain behavior pattern, either overly conservative or introducing too much noise.
All Facts (2)
Reverse KL favors refining the token ranking the student is already active on (mode-seeking), while forward KL is responsible for exploring candidates that the teacher's support covers but the student does not sufficiently attend to (mass-covering).
50%UnverifiedUsing either KL divergence (forward or reverse) alone biases distillation toward a certain behavior pattern, either overly conservative or introducing too much noise.
50%