Unverified50% confidenceSolutionExact time
During SkyRL's weight synchronization, the FP32 router weights are fully preserved to avoid numerical instability caused by low-precision conversion.
1
Sources
50%
Confidence
Long-term
Relevance
10/17/2026
First Seen
Sources
Related Entities
Related Claims
Unverified工程实践中建议将路由器层、注意力投影矩阵及门控网络保留在 BF16/FP16 精度，仅对专家 FFN 权重进行激进量化61% similarUnverifiedRunning 4-bit weights on Hopper or earlier GPUs usually requires dequantizing back to FP8/FP16 before computation, so the compute advantage of FP4 cannot be obtained.58% similarUnverifiedNVFP4 quantization is not 4-bit for the whole model: the vision encoder, connector, Mamba convolutions, and embedding stay in BF16, and the MoE router stays in FP32.58% similarUnverifiedIn mixed-precision training, the forward pass uses FP16 to leverage Tensor Core acceleration while parameter updates retain FP32 to maintain numerical stability, which has become standard practice for large model training.58% similarUnverifiedFP8量化将模型权重从FP16压缩为8位浮点表示，在NVIDIA Hopper架构（H100/H200）上可获得原生加速，且相比INT8对精度损失更小57% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1050657API
curl https://kongchang.com/api/v1/knowledge/claims/1050657MCP
get_claim(id=1050657)