Qwen3.8-27B Local Deployment Benchmarks: Speed Comparison Across RTX 5090, RTX 3090, and Mac with Hardware Buying Guide

Introduction: Is Qwen3.8-27B Really the King of Local Deployment?

Recently, Alibaba's Qwen3.8-27B model has sparked heated discussion on social media. Numerous leaderboards on X (Twitter) claim this model "surpasses Claude 4.6" and even "beats DeepSeek V4." But how true are these claims? Chinese tech YouTuber (Bilibili UP主) KunPeng Talk conducted a systematic evaluation of the model from a hardware benchmarking perspective, covering the RTX 5090, RTX 3090, and Mac M3 Ultra across multiple platforms.

Bottom line up front: Qwen3.8-27B does deliver impressive results in certain specific scenarios like frontend page generation, and it's arguably one of the best-performing models for local deployment right now. However, claims of "comprehensively surpassing Claude/DeepSeek" are significantly overstated, and for individual users, hardware requirements and inference speed remain unavoidable practical concerns.

Qwen3.8-27B Generation Quality Test: Raw Performance in Chatbox

The reviewer first had the model generate a "landing page introducing local deployment of Qwen 3.8" in Chatbox (non-Agent mode). The testing environment was deliberately kept pure — default assistant, no system prompts, no knowledge base, no MCP tools — a pure demonstration of the model's raw capabilities.

It's worth explaining the test environment setup. Chatbox is a popular desktop AI chat client that supports connecting to various local and cloud-based LLMs. The reviewer specifically emphasized testing in "non-Agent mode," and this distinction is crucial. Agent mode refers to having the LLM act as an intelligent agent that autonomously completes complex tasks through tool calling (such as various tools connected via the MCP protocol), knowledge base retrieval, and multi-step reasoning. MCP (Model Context Protocol) is a protocol proposed by Anthropic that allows LLMs to call external tools and data sources in a standardized way. Raw mode strips away all external enhancements, purely testing the model's own understanding and generation capabilities — only by eliminating external variables can you directly measure the model's baseline ability.

The results were impressive. Whether it was the landing page or the subsequently tested "personal resume page," the visual design and code quality exceeded what previous local models could achieve. The reviewer stated outright: "This is probably the best output ever produced by a locally deployed large language model."

One clarification: the video briefly showed "DeepSeek V4 Flash" text because the conversation model was switched later. The landing page results mentioned above were indeed generated by the locally deployed Qwen3.8-27B. This detail also reminds us that screenshots circulating online don't always hold up to scrutiny.

Hardware Inference Speed Benchmarks: Three-Platform Comparison Across 5090, 3090, and Mac

This is the most valuable part of the review. The reviewer answered the core question — "What hardware should you actually use to run Qwen3.8-27B?" — with real data.

Before diving into the hardware comparison, it's necessary to understand the concept of model quantization that's repeatedly mentioned. Quantization refers to compressing model parameters from high-precision floating-point numbers (such as FP32/FP16) to lower-precision representations to reduce VRAM usage and accelerate inference. This review involves three quantization schemes: FP8 is 8-bit floating-point quantization that preserves the exponent and mantissa structure of floating-point numbers with relatively small precision loss; Q4 is 4-bit integer quantization popularized by the llama.cpp ecosystem, widely used in local deployment scenarios; and NVFP4 is NVIDIA's 4-bit floating-point quantization format specifically optimized for its latest Blackwell architecture, requiring specific hardware support. Generally speaking, lower quantization bit-depth means a smaller, faster model, but potentially at the cost of reduced output quality.

RTX 5090: Best Single-Card Inference Speed

On a single RTX 5090, deploying the NVFP4 quantized version through SGLang, the model output speed held steady at 67–68 tokens/second (single-request test), averaging 67.9 tokens/s after 250,000 tokens of cumulative output. This speed is quite impressive for local deployment and essentially reaches a production-ready threshold.

SGLang is a high-performance LLM inference and serving framework developed by UC Berkeley's LMSYS team. Through techniques like RadixAttention, it optimizes KV Cache reuse efficiency and can significantly improve throughput in multi-turn conversation and complex prompt scenarios. Compared to mainstream inference frameworks like vLLM, SGLang has unique advantages in prefix caching and batch scheduling. The reviewer chose SGLang as the deployment solution for the 5090 precisely because of its inference efficiency in single-card scenarios.

RTX 3090: Still Usable with Q4 Quantization

Since the 3090 cannot run NVFP4 (this quantization format depends on Blackwell architecture hardware instructions), the reviewer used the Q4 quantized version instead, with speeds falling in the 40–48 tokens/second range. While not as fast as the 5090, it's still within a usable range.

Mac M3 Ultra 512GB: Unified Memory Advantage Can't Overcome Speed Bottleneck

In response to numerous followers asking "Why not test on Mac?", the reviewer gave a candid answer: It's too slow.

• Running FP8 version via LM Studio: approximately 21 tokens/second
• Running in the reviewer's custom OMX environment: approximately 20 tokens/second, with prefill speed of 109 and thinking time taking up to 23 seconds
• Q4 quantized version: approximately 30 tokens/second, but consuming 37GB of memory

To understand Mac's speed bottleneck, you need to understand Apple Silicon's Unified Memory Architecture (UMA). M-series chips allow the CPU, GPU, and Neural Engine to share the same physical memory. The biggest advantage is that LLMs can utilize all system memory as "VRAM" — the M3 Ultra 512GB version can theoretically load models far larger than what consumer discrete GPUs can handle (the 5090 only has 32GB of VRAM). However, UMA's weakness lies in memory bandwidth: the M3 Ultra's memory bandwidth is approximately 800GB/s, while NVIDIA GPU HBM bandwidth can exceed 1–2TB/s. LLM inference is highly dependent on memory bandwidth (especially during the decode phase when generating tokens one at a time), which is the fundamental reason why Mac can "fit" the 27B model but has inference speeds far inferior to the 5090.

The reviewer specifically pointed out that someone on X claimed M-series Macs can reach 76 tokens/second — "To be honest, I don't believe it." Based on his M3 Ultra testing, the maximum was around 40-something. At speeds around 20 tokens/second, even simple conversations are barely usable, and once the context expands to 512K or even 1M tokens, it becomes practically unusable.

Hardware Buying Advice for Local Deployment: Individual Users Should Think Twice

When asked "What hardware do you recommend for locally deploying Qwen3.8-27B?", the reviewer's stance was very clear: Individual users should not deploy this model.

His specific recommendations are as follows:

• For simple conversations: A single 5090 barely works, but the 5090 currently costs over 30,000 CNY new and over 20,000 CNY used — the value proposition is questionable;
• For running complex Agents: A single card basically won't cut it; dual 4090s or 5090s are recommended;
• If you have less than 32GB of RAM: Forget about it — the Q4 version alone requires around 36GB of memory;
• If you're serious about local deployment: Wait for the Qwen3.8-35B A3B model, which uses MoE architecture with fewer actually activated parameters, making it much more suitable for local deployment.

It's worth elaborating on why MoE architecture is better suited for local deployment. The core idea of MoE (Mixture of Experts) is to split the model's feed-forward network into multiple "expert" sub-networks, with a gating mechanism activating only a few of them during each inference pass. 35B A3B means the model has 35 billion total parameters, but only approximately 3 billion parameters are actually activated per inference (A3B = Active 3 Billion). This allows MoE models to maintain the capabilities of a large model while requiring far less computation and VRAM than a dense model of equivalent parameter count. DeepSeek-V2/V3 adopted a similar strategy, which is why the reviewer recommends individual users wait for the MoE version — the same hardware can deliver better speed and experience.

The Truth About "Surpassing Claude 4.6"

Regarding the most-discussed performance comparison, the reviewer offered an objective assessment:

"In certain areas, it does appear to surpass Claude 4.6 in frontend rendering, but that doesn't mean it does so in all scenarios."

This is key to understanding the current buzz. Any claim of "surpassing" comes with preconditions, and discussing leaderboard rankings without specifying tasks is largely meaningless. What can be confirmed is that compared to Meta's 30B model and Qwen 3.6-27B, Qwen3.8-27B represents a clear step-function improvement in capability.

Additionally, there are several practical issues worth noting:

1. Million-token context is hard to achieve in practice: While the model theoretically supports extending to 1M context, the reviewer's attempts to enable it on SGLang crashed three consecutive times without success. This could be a SGLang bug or a GPU driver issue.

From a technical perspective, implementing million-level context faces severe engineering challenges. The context window determines the maximum text length a model can process in a single inference pass. Scaling from common 4K or 8K to 128K or even 1M tokens hits a core bottleneck: KV Cache VRAM usage grows linearly or even super-linearly with context length. For a 27B parameter model, the KV Cache for 128K context could consume tens of gigabytes of VRAM, while 1M context could require hundreds of gigabytes. Even if the model weights themselves are compressed via quantization to fit within VRAM limits, the additional KV Cache overhead far exceeds single-card hardware capacity. The industry is exploring sparse attention, KV Cache compression, paged management, and other techniques to address this, but achieving million-level context on consumer hardware remains a massive challenge.

2. Not yet available on official channels: Alibaba Cloud's Bailian platform still doesn't list the Qwen3.8 series. Third-party platforms are hosting open-source versions. Although officials claim to offer a 1M context API, it hasn't actually been made available yet.

Conclusion: An Impressive Open-Source Achievement with a High Hardware Barrier

Overall, Qwen3.8-27B is an open-source model with notable technical highlights:

• Strengths: Stunning frontend generation quality, top-tier among local models; substantial improvement over the previous generation; usable speed achievable on a single 5090.
• Weaknesses: High hardware barrier; painfully slow on Mac; million-token context practically unachievable for now; official API not yet available.

For the vast majority of individual users, this model "looks usable, but whether it's actually practical requires hands-on testing." Rather than forcing a 27B model onto marginal hardware now, it's better to wait for the 35B A3B MoE version that's more suitable for local deployment. For developers with 5090/4090-class hardware, it's definitely worth a try.

The reviewer plans to follow up with tests of Qwen3.8-27B's real-world performance in code editors like Cline, as well as speed improvements from Q4+MTP acceleration — stay tuned. MTP (Multi-Token Prediction) is an inference acceleration technique that has the model predict multiple subsequent tokens simultaneously at each inference step rather than generating them one at a time. Combined with methods like speculative decoding, it can significantly boost generation speed without sacrificing output quality, which is crucial for improving user experience in local deployment scenarios.