Unverified50% confidenceFactExact time
Under the square-root learning rate scaling rule, the noise magnitude shrinks as 1/√k as the batch increases, so scaling the learning rate by √k exactly maintains the gradient's signal-to-noise ratio unchanged.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe square-root learning rate scaling rule only preserves the per-sample first-order drift and does not preserve the true convergence rate accumulated over multi-step iteration.70% similarUnverified75倍嵌入学习率是一种保持矩阵不绑定、通过放大嵌入层学习率补偿稀疏更新的可行工程手段，但过高学习率可能导致嵌入空间不稳定震荡69% similarUnverifiedThe cosine annealing learning rate scheduling strategy was proposed by Loshchilov and Hutter in 2017 and smoothly decays the learning rate from its initial value to near zero following a cosine function.68% similarUnverified缩放点积将点积除以√d（d为嵌入维度），使点积方差保持在合理范围，让Softmax输出更平滑的概率分布以稳定训练68% similarUnverifiedWith a fixed learning_rate, when batch_size scales linearly with the number of environments and num_epoch remains unchanged, the total displacement pushing the policy toward a new distribution in a single iteration may increase, manifesting as violent oscillation or even collapse of rewards in the early stages of training.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1027468API
curl https://kongchang.com/api/v1/knowledge/claims/1027468MCP
get_claim(id=1027468)