FP32
FP32 is a 32-bit single-precision floating-point format. It is the baseline precision for deep learning model training and inference, consuming relatively large amounts of storage and computational resources.
Core Facts
Quantizing a model from FP32 to INT8 can improve inference speed by 3–4x, but may lead to reduced detection accuracy for small targets.
The core principle of quantization is to convert model parameters from high-precision floating-point representations (such as FP32's 32-bit floating point) to low-precision representations (such as INT8's 8-bit integer or INT4's 4-bit integer).
Timeline (last 90 days)
During SkyRL's weight synchronization, the FP32 router weights are fully preserved to avoid numerical instability caused by low-precision conversion.
Scoring outputs are more sensitive to numerical perturbations, and low-precision quantization easily causes shifts in score ranking, while ranking accuracy is the core metric for filtering and candidate ranking tasks.
Compute-bound means the hardware's compute units are fully saturated, at which point increasing batch size or optimizing numerical precision (such as using BF16 instead of FP32) becomes meaningful.
FP32 can only maintain approximately 7 significant digits of precision.
FP32 uses 32 bits to represent floating-point numbers, consisting of 1 sign bit, 8 exponent bits, and 23 mantissa bits.
When the cumulative sum reaches 1.0, the smallest change representable by FP32 is 1 ULP = 2⁻²³ ≈ 0.00000012.
The core principle of quantization is to convert model parameters from high-precision floating-point representations (such as FP32's 32-bit floating point) to low-precision representations (such as INT8's 8-bit integer or INT4's 4-bit integer).
Under a 7nm process, a single FP32 multiplication consumes approximately 4.6 pJ of energy, while a single FP32 addition requires only approximately 0.9 pJ.
BF16采用1位符号位、8位指数位和7位尾数位，牺牲精度但保留了与FP32相同的动态范围
All Facts (10)
Quantizing a model from FP32 to INT8 can improve inference speed by 3–4x, but may lead to reduced detection accuracy for small targets.
80%VerifiedThe core principle of quantization is to convert model parameters from high-precision floating-point representations (such as FP32's 32-bit floating point) to low-precision representations (such as INT8's 8-bit integer or INT4's 4-bit integer).
65%UnverifiedDuring SkyRL's weight synchronization, the FP32 router weights are fully preserved to avoid numerical instability caused by low-precision conversion.
50%UnverifiedScoring outputs are more sensitive to numerical perturbations, and low-precision quantization easily causes shifts in score ranking, while ranking accuracy is the core metric for filtering and candidate ranking tasks.
50%UnverifiedCompute-bound means the hardware's compute units are fully saturated, at which point increasing batch size or optimizing numerical precision (such as using BF16 instead of FP32) becomes meaningful.
50%UnverifiedFP32 can only maintain approximately 7 significant digits of precision.
50%UnverifiedWhen the cumulative sum reaches 1.0, the smallest change representable by FP32 is 1 ULP = 2⁻²³ ≈ 0.00000012.
50%UnverifiedFP32 uses 32 bits to represent floating-point numbers, consisting of 1 sign bit, 8 exponent bits, and 23 mantissa bits.
50%UnverifiedUnder a 7nm process, a single FP32 multiplication consumes approximately 4.6 pJ of energy, while a single FP32 addition requires only approximately 0.9 pJ.
50%UnverifiedBF16采用1位符号位、8位指数位和7位尾数位，牺牲精度但保留了与FP32相同的动态范围
50%