RTX 3090 Running Qwen3 27B: Can It Hit 60 Token/s?

Can a Budget RTX 3090 Make a 27B Model Practically Useful?

Can a second-hand RTX 3090 run a 27-billion-parameter model with real-world utility? A Bilibili creator put a 24GB RTX 3090 (Vulcan edition) through its paces on the Qwen3 27B dense model — testing everything from short text and long contexts to multi-user concurrency, and ultimately hooking it up to an Agent framework to build a working tank mini-game. The results were more promising than expected.

Hardware & Model Config: Choosing the Right Quantization for 24GB VRAM

The test platform was an RTX 3090 Vulcan with 24GB VRAM on a GA102 die. The model under test, Qwen3 27B, is a dense model with 27 billion parameters, native support for a 260K-token context window, multimodal (text + image) input, and MTP (Multi-Token Prediction) capability.

The choice of quantization level directly determines whether the model can run at all. For 24GB VRAM, the creator chose Q4 quantization. Those with only 16GB can try Q3 or Q2, but must leave headroom for the KV cache — and output quality will degrade further as the quantization level drops. This is the unavoidable trade-off in local deployment: VRAM, speed, and output quality can't all be maximized at once.

Quantization is the technique of compressing model weights from high-precision floating-point numbers (e.g., FP16/BF16, 2 bytes per parameter) down to low-bit integers. Q4 quantization stores each parameter in just 4 bits, shrinking the theoretical weight size of Qwen3 27B from ~54GB (FP16) to roughly 14–17GB — just enough to fit in 24GB VRAM with room left for the KV cache. Q3/Q2 compress further, but the precision loss leads to more logical errors and increased hallucinations. Popular local inference frameworks like llama.cpp and Ollama both support tiered GGUF quantization; users can download the appropriate quantized version directly from Hugging Face or ModelScope without manual conversion.

Speed Benchmarks: MTP Acceleration Delivers ~60% Gains

Under identical conditions (2,944 input tokens, 1,024 output tokens), the standard mode reached 29.9 token/s; enabling MTP2 pushed that to 48.4 token/s, cutting total generation time from 37.1 seconds to 24.7 seconds.

MTP can be thought of as an inference acceleration algorithm — in this test it boosted sustained generation speed by about 60%. But it's not a free lunch: time-to-first-token is slightly longer, and VRAM usage is higher. This means MTP pays off more on longer outputs; for short Q&A exchanges, the advantage is less pronounced.

MTP (Multi-Token Prediction) is an inference acceleration mechanism. Its core idea is to use additional prediction heads to speculatively guess several future tokens in parallel during standard autoregressive generation. If the guesses are correct, multiple tokens are confirmed in one shot, reducing the number of serial inference rounds. This is conceptually similar to Speculative Decoding, but MTP integrates the auxiliary prediction heads directly into the model architecture — no external draft model needed. MTP2 means a prediction stride of 2, i.e., attempting to output 2 tokens at a time. Because longer speculation steps carry higher verification overhead, short-text gains are minimal. In long-text generation, however, the cumulative hit rate produces a significant speedup — which explains the "longer output = better value" observation in this test.

The Real Long-Context Ceiling

The official spec claims 260K tokens, but how much the hardware can actually handle is another matter. As context length grows, VRAM usage rises and speed drops in both modes. Standard mode completed runs up to 114K tokens; MTP2 completed up to 81K tokens, and failed to finish at 98K tokens.

The creator's practical recommendation is straightforward: keep daily use to around 64K tokens. Official 260K support doesn't mean a single 24GB GPU can comfortably run the full window. This is an especially important reminder for anyone planning local deployment — there's often a significant gap between spec-sheet numbers and real-world usable capacity.

Multi-User Concurrency: Throughput Doubles, But Per-Session Speed Drops

With four concurrent sessions enabled, standard mode achieved a total throughput of 50.3 token/s, while MTP2 hit 125.5 token/s. Note, however, that the MTP2 figure of 125.5 is the combined total across all four sessions — each individual session runs at roughly 36 token/s, not 125.

For scenarios that need to serve multiple requests simultaneously, concurrency meaningfully improves overall throughput efficiency; but from the perspective of any single user's experience, the per-session speed improvement is limited.

Agent Integration: The Local Model Built a Game

The most interesting part was connecting the local model to the DeepSeek^Ad Harness Agent framework. The framework handles file access, terminal commands, and browser control, translating model outputs into real actions. After integration, thinking mode ran at roughly 30 token/s; with thinking mode disabled, peak single-task speed approached 40 token/s.

The creator hit one noteworthy pitfall worth documenting: when using an Agent for long tasks, disable thinking mode first. On the first attempt without disabling it, the model kept generating reasoning chains until the output budget was exhausted — and the task was never actually completed. After turning off thinking mode, the same tank mini-game task finished successfully.

The result was surprising: the local Qwen3 completed the task in 4 minutes 17 seconds, while cloud-based DeepSeek VS Pro took 5 minutes 54 seconds — the local model was over a minute faster.

The creator also cautioned against jumping to leaderboard-style conclusions: this was just one small game-building task, and the more important question is whether the finished product is actually playable — not who won the speed race.

An Agent framework's core function is to bridge the gap between a large language model and external tools: the model handles "thinking and planning," while the framework translates the model's instructions into actual calls against the filesystem, terminal, browser, or APIs — feeding results back to the model and closing the "perceive–decide–act" loop. Thinking Mode is a feature of newer models like Qwen3: when enabled, the model first outputs an extensive internal reasoning process (wrapped in <think> tags) before delivering its final answer. This generally improves accuracy on complex reasoning tasks, but consumes a large portion of the output token budget. In Agent long-task scenarios, the reasoning process alone can fill the maximum single-output limit before any action instructions are even generated — which is exactly the root cause of the pitfall the creator encountered.

Practical Recommendations: Who Is This Setup Right For?

Taking everything together, a 24GB RTX 3090 can run a 27B model with genuine practical utility. The creator's actionable tips are worth saving:

• Routine responses: enable MTP acceleration
• Agent integration: disable thinking mode first to avoid wasting the output budget
• Multi-task scenarios: try four-way concurrency
• Long document processing: try to stay within 64K tokens

This setup isn't for everyone, but if you care about data privacy, prefer a local toolchain, and want to keep costs under control, a single RTX 3090 can get a surprising amount done. For developers and enthusiasts with limited budgets who still want to experience large model local deployment, this is a cost-effectiveness proposition worth taking seriously.