Unverified50% confidenceFactExact time
Conditional entropy H(Y|X) represents the remaining uncertainty of Y given X, and the relationship between entropy and mutual information, H(Y) = I(X;Y) + H(Y|X), is the key identity for decomposing transient and durable contributions.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe concept of perplexity originates from entropy (Shannon Entropy) in information theory, and is essentially the average branching factor per prediction step of a language model.68% similarVerifiedThe semantic entropy method is technically implemented by obtaining candidate answers through multiple samplings, using an NLI model to determine semantic equivalence for clustering, and then computing entropy over the semantic categories.68% similarUnverifiedThe binary cross-entropy loss formula is -[y·log(p) + (1-y)·log(1-p)], where y is the true label and p is the model's predicted probability for the positive class.68% similarUnverifiedThe Shannon entropy of a random variable is defined as H(X) = -∑ p(x) log p(x), measuring the average uncertainty of the random variable.67% similarUnverifiedExcess entropy is defined as the mutual information between the past and the future, measuring in bits the amount of information in the past that is useful for predicting the future.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1010264API
curl https://kongchang.com/api/v1/knowledge/claims/1010264MCP
get_claim(id=1010264)