Unverified50% confidenceFactExact time
The paper 'Exact Tail Laws for Nonlinear SGD with Infinite-Variance Gradients' establishes exact first-order tail probabilities for the last iterate of nonlinear, constant-step-size SGD.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe paper proves that the SGD iterates converge to a stable OU process at the central fluctuation scale and uses this limit to characterize the tail probabilities.72% similarUnverifiedWhen ε ≈ 1, the gradient second-moment term in the denominator is diluted, adaptive scaling has almost no effect, and the optimizer behaves close to pure SGD.64% similarUnverifiedClassical stochastic approximation theory relies on the Robbins-Monro conditions to guarantee convergence, but SGDM with a constant step size does not satisfy these conditions, and the parameters keep fluctuating around a steady state rather than converging to a point.63% similarUnverifiedIn the infinite-variance case t ∈ (1,2), the central fluctuation scale of SGD is of order η^{1/t}, and the limit is a t-stable distribution.63% similarUnverifiedThe Stein identity states that for a function φ that decays sufficiently fast on the boundary, when the probability density p(x) is sufficiently smooth on R^d, E_p[∇_x log p(x)·φ(x) + ∇_x·φ(x)] = 0.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1064973API
curl https://kongchang.com/api/v1/knowledge/claims/1064973MCP
get_claim(id=1064973)