Unverified50% confidenceFactExact time
The Shannon entropy of a random variable is defined as H(X) = -∑ p(x) log p(x), measuring the average uncertainty of the random variable.
1
Sources
50%
Confidence
Long-term
Relevance
8/6/2026
First Seen
Sources
马尔可夫链熵率详解：公式推导与序列建模应用
Related Claims
Partially VerifiedIn information theory, entropy H(p) measures the uncertainty of a probability distribution p, and is equivalent to the average number of bits required to optimally encode information drawn from that distribution.78% similarUnverifiedThe binary cross-entropy loss formula is -[y·log(p) + (1-y)·log(1-p)], where y is the true label and p is the model's predicted probability for the positive class.68% similarUnverifiedEntropy can fully capture information from all moments of a probability distribution, whereas variance can only capture linear, second-order statistical relationships; mutual information can measure total dependence including nonlinear relationships, making it more general than the Pearson correlation coefficient.67% similarUnverifiedFrequentists treat parameters as fixed but unknown constants and seek point estimates by maximizing the likelihood function, while Bayesians treat parameters as random variables and output prediction uncertainty through the posterior distribution.67% similarUnverifiedThe concept of perplexity originates from entropy (Shannon Entropy) in information theory, and is essentially the average branching factor per prediction step of a language model.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/697614API
curl https://kongchang.com/api/v1/knowledge/claims/697614MCP
get_claim(id=697614)