Unverified50% confidenceFactExact time
The vanishing and exploding gradient problems stem from the long-term behavior of the product of Jacobian matrices along time.
1
Sources
50%
Confidence
Long-term
Relevance
11/6/2026
First Seen
Sources
Related Entities
Related Claims
Unverified梯度爆炸的数学本质与链式法则和矩阵连乘的谱范数相关，当雅可比矩阵的最大奇异值持续大于1时，梯度会随层数指数级增长77% similarUnverifiedRepeatedly applying the same set of weights can easily cause gradients to explode or vanish across multiple iterations, similar to the gradient problems of early RNNs.73% similarUnverifiedUnder the assumption that the state transition Jacobian decays exponentially, the error between the truncated gradient and the full-context gradient decays exponentially as the distance from the context boundary increases.73% similarVerifiedRNNs suffer from vanishing/exploding gradient problems, where gradients propagating across a large number of time steps decay or grow exponentially, making it difficult to learn long-range dependencies.71% similarUnverifiedFor finite step sizes, even if a stationary point is unstable under continuous-time gradient flow, if the step size is too large it may skip over the escape direction and cause numerical divergence, while if the step size is too small the escape time may grow exponentially and appear as stagnation.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1141544API
curl https://kongchang.com/api/v1/knowledge/claims/1141544MCP
get_claim(id=1141544)