NF4
A 4-bit data format introduced in the QLoRA paper, designed specifically for neural network weights that follow a normal distribution. NF4 (NormalFloat4) achieves theoretically higher information density than linear INT4 quantization, making it a widely adopted numerical representation for quantizing large language models.
Core Facts
Timeline (last 90 days)
QLoRA builds on LoRA by introducing 4-bit NormalFloat (NF4) quantization, double quantization, and a paged optimizer to compress VRAM usage while maintaining training stability.
Unsloth's Triton kernels dequantize layer by layer on demand, keeping INT4 precise without re-quantizing to NF4, avoiding an approximately 1.6% increase in perplexity.
Re-quantizing INT4 weights to NF4 subjects each weight to two rounds of rounding, and official measurements show this increases the perplexity of Qwen3-8B by 1.6%
NF4 is an information-theoretically optimal 4-bit quantization data type, based on the assumption that pre-trained neural network weights approximately follow a zero-mean normal distribution, dividing the normal distribution CDF into 16 equal intervals as quantization levels.
All Facts (5)
NF4 (4-bit NormalFloat) is an information-theoretically optimal data type proposed in the QLoRA paper (Dettmers et al., 2023).
75%UnverifiedRe-quantizing INT4 weights to NF4 subjects each weight to two rounds of rounding, and official measurements show this increases the perplexity of Qwen3-8B by 1.6%
70%UnverifiedQLoRA builds on LoRA by introducing 4-bit NormalFloat (NF4) quantization, double quantization, and a paged optimizer to compress VRAM usage while maintaining training stability.
50%UnverifiedUnsloth's Triton kernels dequantize layer by layer on demand, keeping INT4 precise without re-quantizing to NF4, avoiding an approximately 1.6% increase in perplexity.
50%UnverifiedNF4 is an information-theoretically optimal 4-bit quantization data type, based on the assumption that pre-trained neural network weights approximately follow a zero-mean normal distribution, dividing the normal distribution CDF into 16 equal intervals as quantization levels.
50%