NVFP4 vs FP8 vs GGUF: 2.3x Faster Decoding on 16GB Ada GPUs

Title

NVFP4 vs FP8 vs GGUF: 2.3x Faster Decoding on 16GB Ada GPUs

Meta Description

Same model Nemotron Dory 1.0 4B compared on RTX 4070 Ti SUPER across NVFP4, FP8, MXFP8, GGUF, and BF16: NVFP4 decodes 2.3x faster, matches BF16 on GSM8K, FP8 leads long-prompt prefill, MXFP8 best token-level fidelity.

Article Content

When you type "nvfp4 vs" into a search engine, the autocomplete options are often "fp8", "gguf", "q4_k_m", and "bf16". The problem with most comparison articles is that they compare different models quantized by different people, with variables not controlled at all. NVIDIA released BF16, NVFP4, MXFP8, and GGUF versions of Nemotron Labs Dory 1.0 4B all at once on October 16, making a controlled comparison possible on the same RTX 4070 Ti SUPER (Ada architecture, 16GB VRAM, no native FP4 hardware). The author wrote down the test rules before the first run and published the pre-registration table and scripts.

Test Setup: Same Model, Same GPU, Pre-registered Rules

The test subject is Nemotron Labs Dory 1.0 4B, for which NVIDIA released BF16, NVFP4, MXFP8, and GGUF formats simultaneously. The hardware is an RTX 4070 Ti SUPER, Ada architecture, 16GB VRAM, without FP4 compute units. This means NVFP4 on this card is not "natively accelerated" but runs via a weight-only Marlin kernel—weights are stored at low precision, while actual computation is still done at 16-bit.

All vLLM formats use the same engine, while GGUF goes through llama.cpp. The author emphasizes that the rules were fixed before the first benchmark run to avoid cherry-picking favorable data after the fact. This pre-registration approach is uncommon in quantization evaluations but directly determines the credibility of the conclusions.

NVFP4 is a 4-bit floating-point weight format introduced by NVIDIA alongside the Blackwell architecture. Each weight occupies only 4 bits, paired with block-level scaling factors (every 16 values share one FP8 scaling coefficient) to preserve dynamic range. Compared to fixed-point formats like INT4, the floating-point structure better represents long-tail values in weight distributions, so at extremely low bit widths it typically has less precision loss than fixed-point quantization. MXFP8 is an 8-bit floating-point format defined by the OCP microscaling standard, also using block-level shared scaling factors, but with a bit width matching FP8, belonging to the "microscaling" family. The key difference between the two: NVFP4 compresses the size to near 3 bits of effective information, while MXFP8 retains 8-bit precision, so the former decodes faster and the latter has higher fidelity—exactly matching the data in the main text.

Weight-only Marlin kernel refers to a GPU kernel that only stores weights at low precision and dequantizes back to 16-bit during computation. Marlin is an efficient implementation in vLLM optimized for 4-bit weights. It allows GPUs without native 4-bit compute (such as Ada architecture) to enjoy the VRAM and bandwidth benefits of reduced weight size, but it cannot reduce the computational load of matrix multiplication, so prefill throughput does not improve. This is the underlying reason why NVFP4 on Ada is "fast at decoding but not advantageous at prefill".

Core Data: Decode Speed Follows Weight Size

Format	Engine	Generation Speed (tok/s)	Read 2048-token Prompt (tok/s)	Weight Size
BF16	vLLM	51.0	7,003	7.69 GiB
FP8 (W8A8)	vLLM	78.5	10,118	4.67 GiB
MXFP8	vLLM	81.4	6,061	4.76 GiB
NVFP4	vLLM	116.8	6,978	3.28 GiB
GGUF Q5_K_M	llama.cpp	58.8	1,710	3.42 GiB

Among the vLLM formats, generation speed roughly follows weight size: NVFP4 has the smallest weights (3.28 GiB) and reaches 116.8 tok/s decoding speed, 2.3x that of BF16. MXFP8 and FP8 have similar sizes and similar speeds. Although GGUF Q5_K_M has a file size close to NVFP4, limited by the llama.cpp engine and different kernel implementations, its generation speed is only 58.8 tok/s.

Long-Prompt Reading: FP8 Is the Winner

Beyond generation speed, prefill performance shows a different pattern. When reading a 2048-token long prompt, FP8 clearly leads at 10,118 tok/s, BF16 is at 7,003 tok/s, NVFP4 at 6,978 tok/s, MXFP8 only at 6,061 tok/s, and GGUF trails at 1,710 tok/s.

The reason lies in the precision path of matrix multiplication: FP8 is the only format among these that allows Ada to perform multiplication at 8-bit. NVFP4 on Ada takes the weight-only path—although weights are small, computation returns to 16-bit, so the prefill phase cannot enjoy the throughput advantage of 4-bit. This explains why NVFP4 has a huge advantage in short-prompt generation scenarios but matches BF16 in long-context reading scenarios.

Prefill is the first stage of inference: the model processes the entire input prompt at once, computing the key-value cache (KV cache) for each token. It is essentially a batch of large matrix multiplications, making it compute-intensive. Decode is token-by-token generation, processing only one token per step, requiring repeated reading of all weights, making it memory-bandwidth-intensive. This distinction explains the pattern in the main text: decode speed follows weight size because smaller weights mean less data to read per step; prefill speed depends on whether the arithmetic precision path can truly reduce computational cost. Although NVFP4 reduces weight size, computation on Ada still returns to 16-bit, so the computational load is unchanged, and prefill is only at BF16 level; FP8 enables Ada to perform multiplication at 8-bit, doubling compute throughput, thus leading in prefill.

Quality Comparison: Four Formats Tie on GSM8K

Beyond speed, the biggest fear with quantization is quality collapse. The author ran the full GSM8K (1319 questions, 8192 token limit): BF16 scored 1249, FP8 scored 1241, MXFP8 scored 1250, NVFP4 scored 1238. Under the ±3 point tolerance rule, all three quantized formats tie with BF16, with 95% confidence intervals all within ±2 points.

This result shows that at the 4B parameter scale, NVFP4's quantization loss does not cause measurable degradation in mathematical reasoning tasks. At least on GSM8K, 4-bit weights do not drag down model capability.

GSM8K (Grade School Math 8K) is a classic benchmark for evaluating mathematical reasoning, containing about 8500 elementary school word problems requiring multi-step arithmetic, typically reported as accuracy or number of correct answers. The "8192 token limit" in the text refers to the maximum length allowed for chain-of-thought reasoning, beyond which it is truncated. GSM8K is relatively sensitive to quantization because it requires the model to maintain multi-step precise calculations; if low precision introduces numerical deviations, errors easily accumulate in intermediate steps. Therefore, still tying with BF16 after quantization on this benchmark is strong evidence that "quantization has not caused substantial quality collapse". However, note that the absolute scores of 4B models on GSM8K are near saturation, which may mask more subtle capability differences—this is also why the author added token-level fidelity tests.

Token-Level Fidelity: MXFP8 Closest to BF16

Overall scores tying does not mean every step's prediction is identical. The author compared token by token on fixed English and Spanish texts: NVFP4 has an 89%–90% probability of choosing the same next token as BF16; FP8 and MXFP8 reach 94%–96%. Perplexity is almost unchanged, but token-level divergence does exist.

On this dimension, MXFP8 is most faithful to BF16. In other words, if your application is sensitive to word-by-word stability of generated content (e.g., structured output, code completion), NVFP4's slight deviations may be more noticeable than GSM8K scores reflect. But if you only look at perplexity and benchmark scores, these differences are masked.

Token-level fidelity measures the frequency with which two models choose the same next token given the same context; perplexity is the exponentiated average log-likelihood over the entire token distribution, reflecting the model's "surprise" at the text. The difference between the two: perplexity is sensitive to the overall shape of the distribution—even if the highest-probability token is chosen incorrectly, as long as the probability mass allocation is similar, perplexity changes little; while token-level agreement only looks at whether the top-ranked token is the same, making it very sensitive to "almost correct" divergences. This is why the 89%–90% vs 94%–96% gap in the text is almost invisible in perplexity but may affect structured output stability—for scenarios requiring word-by-word determinism such as code completion and JSON generation, shifts in the highest-probability token directly manifest as output format errors.

Pitfalls Encountered During Testing

The article also documents problems encountered before measurement: llama.cpp's converter rejected public checkpoints; prefix cache once inflated prefill data; truncated reasoning content was mistakenly counted as answers. The author also mentions that auditing found an error in their own first fidelity test. The value of these details is to remind replicators: the pitfalls of quantization evaluation often lie not in the model itself but in the toolchain and caching behavior.

Practical Recommendations

Based on this data, the selection logic can be summarized as:

• Pursuing generation throughput with tight VRAM: NVFP4 decodes 2.3x faster on Ada with weights of only 3.28 GiB, making it the optimal solution—provided you accept 89%–90% token-level fidelity and the fact of no native FP4 hardware.
• Long prompts, RAG, document Q&A: FP8's 8-bit multiplication gives it a leading prefill speed, with more pronounced benefits in long-context scenarios.
• Needing maximum consistency with BF16: MXFP8 performs best in token-level fidelity.
• GGUF ecosystem: On the same hardware, llama.cpp's prefill speed lags far behind vLLM; if your workflow depends on GGUF, you need to accept this engine-level gap.

The limitations of this test are also clear: single model (4B), single GPU (RTX 4070 Ti SUPER), single inference engine combination. NVFP4 may perform completely differently on platforms with native FP4 hardware such as Blackwell; the weight-only path on Ada is just a transitional solution. But for 16GB Ada users, this set of controlled data provides a rare clear reference.

One-line Summary

Controlled comparison of same model on same GPU: NVFP4 decodes 2.3x faster, FP8 has the strongest prefill, and quality ties with BF16 across the board.

Paragraph Summary

The article leverages NVIDIA's simultaneous release of Nemotron Labs Dory 1.0 4B in BF16, NVFP4, MXFP8, and GGUF formats to conduct a controlled quantization comparison on an RTX 4070 Ti SUPER (16GB Ada, no native FP4 hardware), using pre-registered rules to ensure credible conclusions. Results show: decode speed roughly follows weight size, with NVFP4 weights at only 3.28 GiB and generation reaching 116.8 tok/s, 2.3x that of BF16; however, long-prompt prefill is led by FP8 (10,118 tok/s) because NVFP4 on Ada takes the weight-only Marlin path, with computation still at 16-bit. On full GSM8K, all four formats tie within tolerance; on token-level fidelity, NVFP4 is 89%–90%, with MXFP8 closest to BF16. The article also documents toolchain pitfalls and provides practical recommendations for scenario-based selection.