Unverified50% confidenceFactExact time
Scaling Laws found that the cross-entropy loss of language models follows a power-law relationship with the number of parameters N, dataset size D, and compute C, approximately L ∝ N^(-0.076), L ∝ D^(-0.095), L ∝ C^(-0.050).
1
Sources
50%
Confidence
Long-term
Relevance
8/4/2026
First Seen
Sources
Related Claims
VerifiedCross-entropy loss originates from KL divergence in information theory and measures the degree of difference between the true probability distribution and the predicted probability distribution.63% similarVerifiedCross-Entropy Loss is commonly used for classification tasks, and Mean Squared Error (MSE) is commonly used for regression tasks.61% similarUnverifiedFor a stationary ergodic process, the entropy rate limit exists and equals the conditional entropy limit lim H(X_n|X_{n-1},...,X_1).59% similarUnverified如果交叉熵稳步下降而Dice波动剧烈，问题几乎可以确定出在Dice损失的配置上57% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/684161API
curl https://kongchang.com/api/v1/knowledge/claims/684161MCP
get_claim(id=684161)