Unverified50% confidenceFactExact time
Reverse KL favors refining the token ranking the student is already active on (mode-seeking), while forward KL is responsible for exploring candidates that the teacher's support covers but the student does not sufficiently attend to (mass-covering).
1
Sources
50%
Confidence
Long-term
Relevance
11/3/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedUnlike directly applying cross-entropy (hard labels) to the correction segments, KL supervision uses the teacher's full soft distribution as a soft target, preserving the teacher's relative confidence across candidate tokens, yielding smoother signals and making it less likely for the student to overfit to a single answer.72% similarUnverifiedTeacher intervention pilot studies show that teacher preferences differ across tokens, and that through majority voting and KL-divergence-based selection, a better teacher can be identified for each token.70% similarUnverifiedSeTPO introduces teacher signals only for low-reward trajectories, uses reverse KL divergence to derive token-level teacher feedback, and incorporates it into the policy advantage function.65% similarUnverifiedIn behavior-constrained reinforcement learning, KL regularization can be achieved with access only to the gradient of the reference policy, and the gradient of the reference policy can be obtained through a velocity score transformation.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1129548API
curl https://kongchang.com/api/v1/knowledge/claims/1129548MCP
get_claim(id=1129548)