Unverified50% confidenceFactExact time
KLD is a distribution-level average metric and does not strictly correspond monotonically to downstream task accuracy; a model with slightly higher KLD may happen to score better on a certain type of task.
1
Sources
50%
Confidence
Long-term
Relevance
10/30/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedKL divergence is used to compare the difference in output probability distributions between the fine-tuned model and the original model given the same input; a larger value indicates greater deviation, and a KLD of 0.0244 means the fine-tuning has hardly changed the model's overall behavioral distribution.80% similarUnverifiedKLD (KL divergence) measures the difference between two probability distributions, with the formula D_KL(P||Q)=∑_x P(x)log(P(x)/Q(x)).73% similarUnverifiedKLD measures relative entropy, i.e., the extra number of bits needed to encode the original model's output using the quantized model's distribution, so its unit is bits/token.71% similarUnverifiedKL divergence is an asymmetric metric measuring the difference between two probability distributions, and in reinforcement learning it is used to constrain the degree of deviation between the new and old policies (such as PPO's trust region).68% similarUnverifiedThe scheduling-averaged divergence first computes the KL divergence under each schedule and then takes the expectation, i.e., E_U D_KL(p||q_U).65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1116901API
curl https://kongchang.com/api/v1/knowledge/claims/1116901MCP
get_claim(id=1116901)