Qwen Flash Next IQ4 Quantization Benchmark: Does Low-Bit Quantization Really Lose to FP8?

Among local LLM enthusiasts, debates over quantization formats never seem to end. A widely circulated claim holds that Qwen3 27B in FP8 outperforms Qwen Flash Next in low-bit quantization (such as IQ4). A Reddit user put this to the test using Codex — and the results ran counter to the community's conventional wisdom.

Test Setup and Methodology

This evaluation compared two models: Qwen Flash Next in IQ4_XS quantization (deployed via vllm with r9v) and Qwen3 27B in FP8 (deployed via vllm with radiance).

On the hardware side, the tester used 64GB DDR5 RAM paired with two R9700 GPUs. The benchmarking pipeline was automated through Codex, covering five benchmarks: MMLU-Pro, GPQA Diamond, GSM8K, German MGSM, and IFEval.

It's worth emphasizing that this was not a rigorous, full-scale benchmark run. The tester explicitly noted that MMLU used only 80 randomly sampled questions, and the entire "quick" trial was a subset of just 250 cases. The results reflect trends rather than precise rankings. The author also invited others in the original post to share any more comprehensive evaluations they had conducted.

IQ4_XS is a non-uniform quantization format in the llama.cpp ecosystem, part of the "importance matrix" series (hence the IQ prefix). Unlike traditional uniform quantization formats like Q4_K_M, the IQ series allocates different precision levels based on how important each weight is to the model's output — preserving more bits for critical weights and compressing less important ones more aggressively. This achieves better precision retention at a similar file size. IQ4_XS has an average quantization bit-width of roughly 4.25 bits and is generally considered a "relatively high-quality near-4-bit quantization," though it still falls within the low-bit range. FP8, on the other hand, stores weights and/or activations directly as 8-bit floating-point numbers, with minimal precision loss — much closer to the original FP16/BF16 model, but requiring roughly twice the VRAM and memory compared to 4-bit quantization. The core trade-off is this: FP8 offers higher precision but larger size and is constrained by memory bandwidth; IQ4_XS is smaller and can run faster on lower-bandwidth hardware, but theoretically incurs more precision loss.

Benchmark Results: Flash Next Wins Across the Board

Looking at the results across five benchmarks, Flash Next IQ4_XS outperformed the FP8 27B model in most categories:

Benchmark	Flash-Next IQ4_XS	Qwen3 27B FP8	Winner
MMLU-Pro	83.8%	75.0%	Flash-Next
GPQA Diamond	42.5%	27.5%	Flash-Next
GSM8K	97.5%	90.0%	Flash-Next
German MGSM	92.5%	90.0%	Flash-Next
IFEval	89.6%	89.6%	Tie

The largest gap appeared on GPQA Diamond, where Flash-Next led by 15 percentage points (42.5% vs. 27.5%); MMLU-Pro showed nearly a 9-point advantage as well. Only instruction-following ability (IFEval) ended in a tie.

This result directly challenges the community's common claim that "FP8 27B outperforms low-bit QFN quantization." The tester candidly noted that IQ4_XS is already considered "low-bit quantization," yet neither this benchmark nor their day-to-day work revealed the disadvantages those comments described.

GPQA Diamond is a benchmark designed to test graduate-level scientific reasoning, with questions written by genuine domain experts (physics, chemistry, biology, etc.) and filtered to ensure they cannot be easily answered via search engines. The "Diamond" subset represents the highest-difficulty tier, where average humans score around 35% and even top closed-source models land only in the 60–75% range. This makes it highly sensitive to a model's deep reasoning capabilities — though with small samples, variance can be significant. With only 40 questions sampled here, each question carries roughly 2.5 percentage points of weight, meaning the 15-point gap corresponds to about 6 questions' difference, and statistical significance should be interpreted cautiously. MMLU-Pro is an enhanced version of the original MMLU, featuring more answer choices (10 options instead of 4) and a greater proportion of questions requiring reasoning rather than memorization — making it much harder to inflate scores through guessing or superficial pattern matching.

The Speed and Concurrency Trade-Off

Precision isn't the only consideration. On inference speed, the FP8 27B model showed a clear advantage.

Completing the 250-case "quick" test took the FP8 27B model just 36 minutes and 27 seconds, while Flash-Next IQ4_XS took 2 hours and 5 minutes — roughly 3.5 times slower. With the full evaluation taking 2 hours, the author estimated a more complete next iteration would require over 8 hours, and shelved that plan for the time being.

The tester's practical recommendations are worth noting: for single-user scenarios, even the low-bit quantized Flash Next is the better choice due to its higher accuracy; but for concurrent workloads, FP8 27B runs at roughly 4× the speed of QFN on their system, making it more suitable for multi-user or batch processing tasks.

Limitations and Room for Optimization

This evaluation comes with several caveats. The small sample size (only 80 MMLU questions) means statistical variance could be significant, and individual gaps should not be over-interpreted. The test environment also did not incorporate recent updates to r9v, which may affect Flash Next's actual performance.

Hardware configuration is another variable. The author noted that upgrading to 128GB DDR5 would allow using the mxfp4 quantized version of Flash Next from tcclaviger, which would boost QFN's test speed by approximately 2× — suggesting that the current speed disadvantage of IQ4_XS is partly attributable to memory capacity and quantization format choices, rather than an inherent limitation of the model itself.

mxfp4 is a 4-bit floating-point quantization format based on the MX (Microscaling) specification, jointly standardized by AMD, Intel, Microsoft, and other vendors. Compared to traditional integer 4-bit quantization (such as Q4 or IQ4), MX formats share a single scaling factor across small blocks of weights, theoretically achieving precision closer to floating-point at a similar bit count. AMD's RX 9700 series GPUs have hardware-level acceleration for mxfp4, meaning that on systems equipped with these GPUs, mxfp4 quantized models can take full advantage of hardware instruction set support rather than relying solely on software emulation — which is precisely why the author mentioned that upgrading to 128GB RAM could yield roughly a 2× speed improvement for QFN.

Takeaways for Local Deployment Users

This informal evaluation provides a counterintuitive data point: low-bit quantization does not necessarily mean a significant capability regression. Differences in model architecture and scale can sometimes outweigh the impact of quantization precision. Flash Next, as a newer architecture, still outperforms the larger 27B FP8 model even at IQ4_XS.

For local deployment users, the right choice depends on usage patterns: prioritize low-bit Flash Next for single-session quality; choose FP8 27B for throughput and concurrent workloads. As for the conventional wisdom circulating in the community, it's best verified on your own hardware with your actual workload rather than taken at face value.

The author also extended an open invitation to the community: does anyone have more representative benchmarks to recommend, or has anyone conducted a more comprehensive comparison? User-driven cross-evaluations like this represent some of the most valuable practical knowledge the local LLM community has to offer.