Partially Verified65% confidenceFactExact time
KL divergence D(p,q) describes how many extra bits are wasted when using an incorrect distribution q to encode data that actually follows distribution p, compared to using the correct distribution p.
3
Sources
65%
Confidence
Long-term
Relevance
8/2/2026
First Seen
Sources
From Entropy to Cross-Entropy: An Information-Theoretic Derivation of the Logistic Regression Loss Function
Related Entities
Related Claims
Unverified从KL散度角度看，每一代合成数据与真实分布之间的散度都在单调递增，且误差会在迭代中累积74% similarUnverifiedIn Dreamer V1, the KL divergence term is used to constrain the distance between the posterior distribution q(z_t|h_t,o_t) and the prior distribution p(z_t|h_t); if the KL weight is too large, the posterior is forced to approach the prior, causing the latent variables to lose observational information (information loss), and if the weight is too small, the transition model cannot generate meaningful imagined trajectories from the prior (posterior collapse).69% similarUnverifiedThe exponential distribution Exp(λ) describes the waiting time between successive events in a Poisson process and possesses the memoryless property; the Gamma distribution is the distribution of the sum of k independent exponential random variables and degenerates to the exponential distribution when k=1.67% similarUnverifiedWhen conditional dependencies exist between positions, the KL divergence between the product distribution and the true joint distribution is strictly greater than zero, and it increases as dependencies grow stronger and more positions are written simultaneously.66% similarUnverifiedWhen the data distribution is balanced, D* approaches a certain baseline value, and the new formula reduces to a result close to the traditional formula; when the data are severely imbalanced or skewed, D* deviates significantly from the baseline, leading to large adjustments in the estimate of predictive performance.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/679894API
curl https://kongchang.com/api/v1/knowledge/claims/679894MCP
get_claim(id=679894)