Unverified50% confidenceFactExact time
Shannon's information entropy formula is H = -Σ p(x) log₂ p(x), which describes the average uncertainty of a random variable.
1
Sources
50%
Confidence
Long-term
Relevance
8/31/2026
First Seen
Sources
Related Entities
Related Claims
Unverified不确定性估计主要有两条技术路径：利用Token概率（logprobs）和蒙特卡洛采样（对同一输入进行5-20次高温度采样并计算香农熵）73% similarUnverifiedFor a stationary ergodic process, the entropy rate limit exists and equals the conditional entropy limit lim H(X_n|X_{n-1},...,X_1).66% similarUnverified从信息论角度，DSL约束的本质是降低了输出分布的熵，一个等概率的N选1分布熵值为log₂N66% similarVerifiedCross-entropy loss originates from KL divergence in information theory and measures the degree of difference between the true probability distribution and the predicted probability distribution.64% similarUnverified逻辑回归中的交叉熵损失如果直接对 log(0) 求值会产生负无穷，需要加入极小的 epsilon 值或使用 log-sum-exp 技巧63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/829567API
curl https://kongchang.com/api/v1/knowledge/claims/829567MCP
get_claim(id=829567)