NVFP4
NVFP4 is a 4-bit floating-point (FP4) numerical quantization format introduced by NVIDIA for the Blackwell architecture, representing a low-precision numerical standard. Designed for deep learning inference and training scenarios, it reduces memory usage and computational overhead by lowering the bit width of numerical representations. Compared to conventional integer quantization formats, NVFP4 offers advantages in numerical range coverage and precision loss control, and is primarily used for efficient deployment and accelerated computation of large-scale neural network models.
Core Facts
NVFP4 is a 4-bit floating-point (FP4) quantization format introduced by NVIDIA, designed specifically for the hardware acceleration capabilities of Blackwell architecture GPUs
NVIDIA designed a dedicated Tensor Core instruction mma.kind::mxf4 for NVFP4 in the Blackwell architecture (SM 100/103, such as the B100 and B200 series), and adopted an FP8 scaling factor mechanism with one scale per group of 16 elements.
In an RTX 5070 (12GB VRAM) configuration, the local Qwen3 9B model's output speed reached approximately 70–76 tokens/second.
Timeline (last 90 days)
Ollama 0.19 was released as a preview, supports only one model, requires Mac memory greater than 32GB, and the new test used a different NVFP4 format.
NVFP4 is actually approximately 3.75 bits, and is not strictly a 4-bit quantization in the rigorous sense.
Cosmos comes in three sizes ranging from 4 billion to 64 billion parameters, is fully multimodal, commercially usable, and customizable, with its Super NanoEdge version optimized for NVFP4 and speculative decoding.
The competition programming model is fine-tuned from Nemotron-3-Ultra-550B-A55B and uses the NVFP4 low-precision quantization format.
The difference between NVFP4 and INT4 is that NVFP4 retains the dynamic range expressiveness of floating-point numbers, making it more friendly to large models with non-uniform weight distributions.
According to NVIDIA's official tests, the optimal decoding combination for Qwen3 27B on Jetson is NVFP4 quantization combined with speculative decoding.
NVFP4 is typically used in conjunction with fine-grained scaling factors (per-group scaling) and activation-aware quantization (AWQ/GPTQ) to offset quantization error.
Primitive has provided a Flash Next model page on Hugging Face with mixed NVFP4 and FP8, along with a companion Docker image and model, using vLLM as the inference engine.
NVFP4 is a 4-bit floating-point quantization format used to reduce memory usage and improve inference throughput while preserving model accuracy.
NVFP4 is originally an NVIDIA proprietary format, and AMD's support for this format on the MI355 means that native hardware-level computation support for this numeric format has been added.
21 more timeline events
All Facts (20)
NVFP4 is a 4-bit floating-point (FP4) quantization format introduced by NVIDIA, designed specifically for the hardware acceleration capabilities of Blackwell architecture GPUs
90%VerifiedNVIDIA designed a dedicated Tensor Core instruction mma.kind::mxf4 for NVFP4 in the Blackwell architecture (SM 100/103, such as the B100 and B200 series), and adopted an FP8 scaling factor mechanism with one scale per group of 16 elements.
75%VerifiedIn an RTX 5070 (12GB VRAM) configuration, the local Qwen3 9B model's output speed reached approximately 70–76 tokens/second.
70%VerifiedNVFP4 is a 4-bit floating-point format (FP4) introduced by NVIDIA, using floating-point representation (including sign bit, exponent bit, and mantissa bit) to cover the numerical space in a non-uniform manner.
65%VerifiedThe Unsloth team released a new Qwen3.8-27B GGUF model based on the Dynamic v3 quantization scheme, achieving an additional 10% improvement in top-1% accuracy while maintaining the same model size.
85%UnverifiedNVFP4量化需要50系列NVIDIA显卡才能使用
85%Unverified在非50系显卡上使用NVFP4会退化为软件模拟，导致显存占用大、精度差，无任何优势
85%UnverifiedNVFP4量化在Blackwell架构上相比FP16吞吐量提升约40-50%
80%UnverifiedOllama 0.19 was released as a preview, supports only one model, requires Mac memory greater than 32GB, and the new test used a different NVFP4 format.
50%UnverifiedNVFP4 is actually approximately 3.75 bits, and is not strictly a 4-bit quantization in the rigorous sense.
50%UnverifiedCosmos comes in three sizes ranging from 4 billion to 64 billion parameters, is fully multimodal, commercially usable, and customizable, with its Super NanoEdge version optimized for NVFP4 and speculative decoding.
50%UnverifiedThe competition programming model is fine-tuned from Nemotron-3-Ultra-550B-A55B and uses the NVFP4 low-precision quantization format.
50%UnverifiedNVFP4 is typically used in conjunction with fine-grained scaling factors (per-group scaling) and activation-aware quantization (AWQ/GPTQ) to offset quantization error.
50%UnverifiedAccording to NVIDIA's official tests, the optimal decoding combination for Qwen3 27B on Jetson is NVFP4 quantization combined with speculative decoding.
50%UnverifiedThe difference between NVFP4 and INT4 is that NVFP4 retains the dynamic range expressiveness of floating-point numbers, making it more friendly to large models with non-uniform weight distributions.
50%UnverifiedPrimitive has provided a Flash Next model page on Hugging Face with mixed NVFP4 and FP8, along with a companion Docker image and model, using vLLM as the inference engine.
50%UnverifiedNVFP4 is originally an NVIDIA proprietary format, and AMD's support for this format on the MI355 means that native hardware-level computation support for this numeric format has been added.
50%UnverifiedNVFP4 is a 4-bit floating-point quantization format used to reduce memory usage and improve inference throughput while preserving model accuracy.
50%UnverifiedWhen running the Qwen3 27B NVFP4 quantized model on a single AMD Radeon R9700, performance doubled across all metrics after optimization.
50%UnverifiedThe test was conducted based on the Qwen3 27B NVFP4 quantized version released by Unsloth.
50%