Unverified50% confidenceFactExact time
Softmax Collapse causes the gradient of the target logit to be rounded to zero, breaking the property that gradients sum to zero.
1
Sources
50%
Confidence
Long-term
Relevance
10/23/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedWithout weight decay, unbounded logit growth causes numerical underflow in standard softmax, making gradients vanish entirely and manifesting as a sudden accuracy collapse.76% similarUnverified在高维空间中通过令导数为零求解析解几乎不可能，梯度下降是可扩展到高维的策略68% similarUnverifiedThe gradient imbalance caused by Softmax Collapse may lock the model prematurely into a memorization solution, delaying or preventing the transition to a generalization solution.67% similarUnverifiedUnder weight-free gradient flow, any stationary point whose MMD value is far from zero is unstable, and gradient flow will not stably remain at these bad stationary points.66% similarUnverified当推理延迟 τ = d_max/2 时，可行执行视界窗口坍缩为零，不存在可行解，称为延迟墙。65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1088264API
curl https://kongchang.com/api/v1/knowledge/claims/1088264MCP
get_claim(id=1088264)