Qwen 3.8 Flash Next Hands-On: Single-GPU Deployment That Outperforms DeepSeek V4?

Yesterday, Qwen went open source once again, releasing Qwen 3.8 Flash Next — a Flash model built on the next-generation Qwen 4 architecture, with full weights available on launch day. For local deployment enthusiasts, this is undoubtedly a pleasant surprise. This article is based on a comprehensive hands-on review by Bilibili creator bwd6, diving deep into the model's architectural innovations, deployment strategies, and its head-to-head showdown with the recently released DeepSeek V4 Flash.

A New Architecture: Trading Space for Compute with Ngram Design

Qwen 3.8 Flash Next features an entirely new multimodal Mixture of Experts (MoE) architecture. The main model has 125 billion total parameters but activates only 6 billion during inference.

Mixture of Experts (MoE) is a sparsely activated neural network architecture. Its core idea is to distribute model parameters across multiple "expert" sub-networks, with a gating network (Router) selectively activating only a handful of experts to process each input during inference. This means a model can have an enormous total parameter count (which determines its knowledge capacity ceiling) while requiring far less computation per inference than a dense model of equivalent size. Qwen 3.8 Flash Next's 125B total / 6B active parameter split implies a sparsity ratio of roughly 95% — only about 5% of parameters are used per inference, making it one of the most aggressively sparse MoE models currently available. For comparison, Mixtral 8x7B has an activation rate of about 25%, and DeepSeek-V2 sits at roughly 10%. Lower activation rates yield faster inference but impose stricter demands on the expert routing strategy to avoid "expert collapse" — the problem where most inputs are consistently routed to just a few experts.

The biggest highlight is the addition of a 51 billion parameter Ngram module — think of it as a massive local phrase memory table. Ngram (N-gram) is one of the most classic statistical models in natural language processing, modeling local language patterns by counting how frequently consecutive N tokens co-occur. In Qwen 3.8 Flash Next, the Ngram module takes on an entirely new meaning — it's no longer a traditional statistical frequency table but rather a deeply trained, 51-billion-parameter giant lookup structure. It works similarly to a "draft model" in Speculative Decoding: when the main MoE model needs to generate the next token, the Ngram module first makes a quick prediction based on local phrase patterns in the current context. If the prediction hits (i.e., the main model validates it), the full forward pass of the main model can be skipped entirely, and the Ngram's prediction is adopted directly. This mechanism is especially effective for high-frequency phrases, fixed collocations, code templates, and other highly repetitive content.

The core philosophy behind this phrase memory table is "trading space for compute." When the model encounters common phrases and local patterns, it can simply look up results rather than recompute them. As a result, its computational load is noticeably lower than other models with similar parameter counts. The trade-off, of course, is direct: you need an additional 50GB to 100GB of RAM or VRAM to host the Ngram table.

After the model's release, local deployment enthusiasts are most concerned about three questions:

1. Can the Ngram module be offloaded to system RAM or even SSDs? Major inference frameworks are actively working on RAM offloading support for Ngram, but there's no official path for SSD-resident Ngram yet.
2. Can devices with 96GB or 128GB unified memory run it reliably? Many users with such hardware (e.g., Apple Silicon Macs) are eagerly awaiting mature solutions.
3. Can it actually beat DeepSeek V4 Flash in real-world performance? This is the question most people care about.

Local Deployment in Practice: Running the Full Setup on a Single 96GB GPU

The test was conducted on a rented machine with 96GB of VRAM and 188GB of system RAM, using embedding table offloading to system memory so that VRAM and RAM work together for hybrid inference.

The inference framework of choice was vLLM, as it's currently the only one capable of running the full setup on SM120 GPUs. vLLM is a high-performance LLM inference and serving framework developed by UC Berkeley's Sky Computing Lab. Its core innovation is PagedAttention — borrowing the paging concept from operating system virtual memory management, it splits the KV Cache into fixed-size pages for dynamic allocation, dramatically reducing memory fragmentation and waste. In this test scenario, vLLM also demonstrated another critical capability: offloading certain model parameters (such as embedding tables and the Ngram module) to system memory for hybrid VRAM-RAM inference.

SM120 refers to the streaming multiprocessor designation in NVIDIA's latest-generation Blackwell GPU architecture. Each generation of NVIDIA GPUs updates its SM version number: Ampere is SM80/SM86, Hopper is SM90, and Blackwell is SM100/SM120. Because the architecture is so new, many inference frameworks' CUDA kernels haven't been adapted or optimized for SM120 yet. SGLang currently doesn't support this GPU and crashes during warmup; llama.cpp still has optimization pull requests pending. After roughly two hours of debugging, the creator successfully got the model running on a single 96GB GPU.

Real-world testing revealed that while the Ngram module must reside in system memory, the impact on inference speed is negligible. With MTP (Multi-Token Prediction) enabled, generation speed across various context lengths consistently stayed above 100 tokens per second, while the prefill stage exceeded 10,000 tokens per second across the board. MTP is an important direction in LLM inference acceleration — traditional autoregressive language models predict only one token at a time, whereas MTP allows the model to predict multiple consecutive tokens in a single forward pass. The Ngram module in Qwen 3.8 Flash Next is essentially an engineering implementation of the MTP concept: the Ngram table rapidly provides multiple candidate tokens, which the main model then validates in parallel, boosting the effective output per inference step from one token to several. This explains why enabling MTP pushes generation speed past the 100 tokens-per-second threshold.

It's worth explaining that LLM inference consists of two fundamentally different stages: Prefill and Decode. The Prefill stage processes all input prompt tokens at once — since input tokens can compute attention in parallel, this stage is primarily compute-bound. The Decode stage generates output tokens one at a time, with each new token requiring a full read of the KV Cache, making it primarily memory-bandwidth-bound. The fact that prefill speed exceeds 10,000 tokens per second while decode speed is 100+ tokens per second — a hundredfold gap — is a textbook illustration of the different bottleneck characteristics of these two stages.

As the creator put it, "Once deployment was done, the speed felt absolutely insane."

Even more noteworthy: compared to DeepSeek V4 Flash, this setup saves an entire GPU while actually being faster. For local deployment enthusiasts, this is an extremely attractive value proposition.

Eight-Benchmark Showdown: Qwen vs. DeepSeek V4 Flash

The evaluation used eight custom LLM benchmarks created by the reviewer, pitting Qwen against the just-released DeepSeek V4 Flash Vision experimental version (nicknamed "Little Whale"). Note that the DeepSeek experimental version is currently only accessible via its official API — weights have not been released.

Language and Reasoning

• Chinese Writing: Qwen scored 7, Little Whale scored 8. Qwen's writing style felt "flat" and unremarkable.
• Logical Reasoning (alien deity puzzle): Both achieved perfect scores with no difficulty.

Multimodal and Vision

• Invoice Recognition: Qwen got all 28 data fields correct for a perfect 10; Little Whale missed one field and scored 9.
• Long-Context Needle Test: Both accurately located the target information and completed reasoning — both scored perfectly.
• Advanced Image Understanding: Qwen scored perfectly — "you can always trust Qwen's image recognition capabilities." Little Whale made two detail errors and scored 8. The V4 Flash Vision experimental model still falls short of top-tier accuracy in image understanding.
• Office Applications (understanding 171 multimodal files and organizing them into a table): Both scored perfectly.

Long-Form Complex Tasks

The seventh benchmark is the most demanding test of a model's comprehensive capabilities: processing roughly a quarter's worth of raw materials, writing to a database, building backend APIs, and delivering a complete full-stack application.

Qwen 3.8 took 51 minutes and consumed 19 million tokens. Points were deducted mainly for missing dark mode and some layout imperfections. Little Whale finished in 41 minutes but consumed 70% more tokens than Qwen, had more layout issues, and achieved lower feature completeness.

The standout achievement for Qwen 3.8 Flash Next is scoring the second-highest on the full-stack application task while using significantly fewer tokens than other models — making it an exceptionally efficient model. "Efficient" here carries a dual meaning: on one hand, fewer tokens mean lower API call costs; on the other, it indicates the model is more precise at understanding and executing complex instructions, without needing verbose "thinking" processes to compensate for comprehension gaps.

Final Rankings and Price Advantage

Among all local models the reviewer has tested, Qwen 3.8 Flash Next ranked second, falling just one point short of its sibling 27B model (though the 27B version used the full-precision BF16 unquantized weights). Little Whale trailed by two spots in fourth place, though it remains an experimental version — the reviewer plans to retest once DeepSeek releases the full weights.

Note: This ranking only reflects model performance in a specific local testing environment and is provided for reference only — it should not be used as a basis for any decisions.

On pricing, Qwen also demonstrates clear competitiveness.

Assuming 1 million input tokens, 50,000 output tokens, and a 99% cache hit rate: Qwen's overall cost is roughly 60% of DeepSeek's off-peak pricing and 30% of its peak pricing. Modern LLM API services widely use "Prefix Caching" to reduce the cost of redundant computation — when multiple requests share the same prompt prefix (e.g., system prompts, long document context), the server can reuse previously computed KV Cache, eliminating redundant prefill calculations. This is why cached input pricing is significantly cheaper than regular input pricing. A 99% cache hit rate is a typical scenario for agent tasks: during multi-step execution, each step carries the same system prompt and full conversation history, with only a small portion of new input added. In such scenarios, cache hit pricing essentially determines the total cost.

For local agent task scenarios, where tasks repeatedly read large amounts of context, cache hit pricing is the dominant long-term cost factor — making the actual price gap even more dramatic than spreadsheet calculations suggest.

Conclusion: The Shifting Landscape of Multimodal Flash Models

From performance to pricing, both Qwen and DeepSeek are visibly ramping up their investment in multimodal Flash models. Just yesterday, Zhipu AI also released its first multimodal Flash model, officially entering the race. It's safe to predict this segment will only get more competitive.

The term "Flash model" is a product positioning concept that has emerged in the industry over the past six months, referring to model versions that maintain near-flagship capabilities while dramatically optimizing inference speed and cost. Google's Gemini Flash series pioneered this category, and other vendors have quickly followed suit. Competition among Flash models is fundamentally a competition over efficiency — given a fixed compute budget, whoever delivers better performance gains an advantage in API pricing and local deployment accessibility.

For local deployment enthusiasts, Qwen 3.8 Flash Next — with its efficient Ngram-powered inference, single-GPU deployment friendliness, and solid multimodal capabilities — genuinely has the potential to serve as a viable alternative to DeepSeek V4 Flash. This is especially true when you need stable, openly available weights and a fully self-controlled local environment.