Unverified50% confidenceFactExact time
The cosine annealing learning rate scheduling strategy was proposed by Loshchilov and Hutter in 2017 and smoothly decays the learning rate from its initial value to near zero following a cosine function.
1
Sources
50%
Confidence
Long-term
Relevance
9/1/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedCosine annealing allows the learning rate to smoothly decrease along a cosine curve to near zero and periodically restart, which was formalized by the SGDR algorithm.77% similarUnverified梯度下降的收敛条件与学习率和损失函数的Lipschitz常数的关系有关72% similarUnverified梯度下降的权重更新规则为 w = w - lr * gradient，其中lr是学习率68% similarUnverifiedThe parameter update rule for gradient descent is θ_{t+1} = θ_t - η·∇L(θ_t), where η is the learning rate, and updating parameters in the direction of the negative gradient reduces the loss function most rapidly.66% similarUnverified75倍嵌入学习率是一种保持矩阵不绑定、通过放大嵌入层学习率补偿稀疏更新的可行工程手段，但过高学习率可能导致嵌入空间不稳定震荡64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/837582API
curl https://kongchang.com/api/v1/knowledge/claims/837582MCP
get_claim(id=837582)