Unverified50% confidenceFactExact time
When cross-entropy loss is combined with Softmax, the output layer gradient takes the form ∂L/∂z = ŷ - y, which avoids the vanishing gradient problem of MSE when the output layer is saturated.
1
Sources
50%
Confidence
Long-term
Relevance
11/2/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedCross-entropy loss often converges faster than MSE in classification tasks because, when paired with softmax output, gradients do not vanish in the saturation region.80% similarUnverified在计算交叉熵损失时，softmax对正确标签位置i的梯度为p_i-1，对其他位置j的梯度为p_j，使lm_head输出层梯度密集72% similarUnverifiedMSE suffers from gradient saturation when Sigmoid outputs are close to 0 or 1, causing training to be extremely slow, whereas cross-entropy has larger gradients when predictions deviate severely68% similarUnverifiedWhen ε ≈ 1, the gradient second-moment term in the denominator is diluted, adaptive scaling has almost no effect, and the optimizer behaves close to pure SGD.66% similarUnverifiedThe paper controls the convergence rate of the NPMLE by sharply characterizing the bracketing entropy of finite Gaussian mixture classes under local Hellinger distance.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1126398API
curl https://kongchang.com/api/v1/knowledge/claims/1126398MCP
get_claim(id=1126398)