ConceptFP8 / 8位浮点量化
FP8 Quantization
A quantization technique that compresses model parameters from FP16/FP32 precision to an 8-bit floating-point representation. By retaining floating-point exponent bits, it preserves model accuracy more effectively than INT8 integer quantization, reducing memory usage by approximately 50% and improving inference throughput.
Timeline (last 90 days)
Oct 19
The FP8-quantized GLM-5 shows significantly higher gains, and one reasonable explanation is that an FP8 model has higher single-prefill compute throughput and the absolute time of repeated prefill accounts for a larger share, so the benefit of eliminating recomputation is also more pronounced.
Unverified50%