Unverified50% confidenceFactExact time
Prior work has shown that on homogeneous models, when the learning rate schedule is of the strongly decaying form η(t) = o(t^{1/L-1}), Adam is proven to converge to the solution that maximizes the ℓ∞ margin.
1
Sources
50%
Confidence
Long-term
Relevance
11/3/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAdam's bias correction is performed by dividing the first and second moments by (1-β1^t) and (1-β2^t) respectively, where t is the time step and β1 and β2 are the decay rates.76% similarUnverifiedAdamW's effective step size is approximately proportional to η/(1-β₁), so simply increasing β₁ amounts to implicitly amplifying the effective learning rate.74% similarUnverifiedWhen ε ≈ 0, the second-moment estimate fully dominates and the optimizer reverts to standard Adam.72% similarUnverifiedThe paper establishes a loss decay rate for full-batch Adam on general homogeneous models, whose leading order depends only on the second-moment parameter.71% similarUnverifiedAdam的一阶矩衰减系数β₁通常默认为0.970% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1129590API
curl https://kongchang.com/api/v1/knowledge/claims/1129590MCP
get_claim(id=1129590)