Unverified50% confidenceFactExact time
In the control experiment where the NVFP4 error is doubled, noise injected only into the Adam second-moment estimate behaves like a reduced learning rate, and its cost gradually fades over training; noise injected into the update reduces the signal-to-noise ratio, carries about two-thirds of the cost, and persists until the end of training.
1
Sources
50%
Confidence
Long-term
Relevance
11/10/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIncreasing the learning rate to compensate for the smaller average step size failed to recover the rare-fact learning cost caused by quantization noise or by NVFP4 itself.79% similarUnverifiedIn the sampled-gradient setting, when the learning rate η decreases, the expected gain under fixed β vanishes faster than the fluctuation itself, causing the benefit of momentum to be swallowed by noise.75% similarUnverifiedAction latency and noise in the physical system further reduce sample efficiency, making training cycles that were feasible in simulation impractical.73% similarUnverified延迟解耦时如果只复制权重而重置优化器动量状态,Adam会经历bias correction阶段导致数百步训练效率损失,并可能引发训练损失突然跳升73% similarUnverifiedRe-running Quiet-STaR's training recipe with an optimizer capable of applying every update caused model performance to diverge; the high-precision optimizer required lowering the learning rate by 3 to 10 times to obtain the same results and could not achieve better results.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1154471API
curl https://kongchang.com/api/v1/knowledge/claims/1154471MCP
get_claim(id=1154471)