Unverified50% confidenceConceptExact time
In backpropagation, gradient values are often extremely small, and under low precision they easily underflow or are drowned out by rounding noise, requiring techniques such as loss scaling, an FP32 master weight copy or stochastic rounding to maintain numerical stability.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedLow-bit representation significantly amplifies the impact of outliers in weight distributions; SmoothQuant addresses this by migrating quantization difficulty, while SpQR addresses it by separately retaining high precision for outliers.75% similarUnverifiedThe deconvolution problem can be formulated as a division operation in the frequency domain, and its difficulty depends on the decay rate of the characteristic function of the error distribution74% similarUnverifiedFP8动态范围较窄，在反向传播和梯度累积时容易出现数值溢出或下溢，因此训练侧actor保持BF16精度74% similarVerified混合精度训练将权重保存为FP32，而前向和反向传播计算使用FP16或BF16，可将显存占用减少近一半73% similarUnverifiedA similarity threshold that is too high may cause valid answers to be incorrectly filtered out, while one that is too low may introduce noise, requiring a trade-off between recall and precision.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1026578API
curl https://kongchang.com/api/v1/knowledge/claims/1026578MCP
get_claim(id=1026578)