Unverified50% confidenceFactExact time
The hardware bottleneck of backpropagation is not compute power but memory access and precision; the activation values of each layer must be retained until gradient backpropagation is complete, and an FP32-precision gradient accumulation buffer is usually required.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn backpropagation, gradient values are often extremely small, and under low precision they easily underflow or are drowned out by rounding noise, requiring techniques such as loss scaling, an FP32 master weight copy or stochastic rounding to maintain numerical stability.73% similarVerified混合精度训练将权重保存为FP32，而前向和反向传播计算使用FP16或BF16，可将显存占用减少近一半72% similarUnverifiedFP8动态范围较窄，在反向传播和梯度累积时容易出现数值溢出或下溢，因此训练侧actor保持BF16精度70% similarUnverifiedWithout calibrating on real hardware and instead using constants derived directly from a simulator, the router loses 4.5 goodput points and approximately 40% of its tail-latency advantage.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1032394API
curl https://kongchang.com/api/v1/knowledge/claims/1032394MCP
get_claim(id=1032394)