ConceptE4M3 / FP8 E4M3

FP8 E4M3

An 8-bit floating-point format with a 4-bit exponent and a 3-bit mantissa; it offers a narrower dynamic range but higher precision, making it suitable for scenarios with concentrated numerical distributions such as the KV cache.

Timeline (last 90 days)

Oct 3

DeepEP compresses Token data from BF16 to FP8 E4M3, halving the volume, and attaches a scale factor to each Token for restoration, which constitutes per-token quantization granularity.

Unverified50%

All Facts (1)

Unverified

DeepEP compresses Token data from BF16 to FP8 E4M3, halving the volume, and attaches a scale factor to each Token for restoration, which constitutes per-token quantization granularity.

50%

Source Articles

One Parameter Fits 128K Context on a 4090: FP8 KV Cache Quantization in PracticeOct 19