Unverified50% confidenceFactExact time
For a stationary ergodic process, the entropy rate limit exists and equals the conditional entropy limit lim H(X_n|X_{n-1},...,X_1).
1
Sources
50%
Confidence
Long-term
Relevance
8/6/2026
First Seen
Sources
马尔可夫链熵率详解：公式推导与序列建模应用
Related Claims
Unverified跨分词器蒸馏前应先测量熵保留率或概率质量对齐度以避免在错误目标上浪费算力64% similarUnverified如果交叉熵稳步下降而Dice波动剧烈，问题几乎可以确定出在Dice损失的配置上63% similarUnverified逻辑回归中的交叉熵损失如果直接对 log(0) 求值会产生负无穷，需要加入极小的 epsilon 值或使用 log-sum-exp 技巧61% similarUnverified从信息论角度，DSL约束的本质是降低了输出分布的熵，一个等概率的N选1分布熵值为log₂N60% similarUnverifiedScaling Laws found that the cross-entropy loss of language models follows a power-law relationship with the number of parameters N, dataset size D, and compute C, approximately L ∝ N^(-0.076), L ∝ D^(-0.095), L ∝ C^(-0.050).59% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/698768API
curl https://kongchang.com/api/v1/knowledge/claims/698768MCP
get_claim(id=698768)