Testing a Local 27B Model: How Complex Can Qwen3.8 Go with Real Coding Tasks?

Local LLMs: Not a Replacement, But a Complement

An overseas developer blogger ran a hands-on evaluation of the newly released open-source model Qwen3.8 (27 billion parameters). Unlike typical benchmarking approaches, he made his stance clear from the start: he wasn't trying to determine whether this model could replace top-tier cloud models like Opus or Gemini — that kind of direct comparison simply isn't fair.

The real question he cared about was: How complex of a real-world task can a locally-run model at this size reliably handle?

In his view, local models aren't full replacements for paid cloud models — they're a complementary tool, suited for relatively simple and routine tasks, as well as scenarios where you don't want your data leaving your local machine. This framing runs throughout the entire evaluation and makes the conclusions feel grounded and practical.

Hardware Requirements: VRAM Is Everything

The blogger admitted that his daily driver is a MacBook with only 24GB of unified memory — and while Qwen3.8 can technically run on it, it's too slow to be usable. So he deployed the model on a desktop machine and accessed it remotely from his MacBook over the local network.

His core hardware advice is straightforward: If you want to run Qwen3.8, the most important spec is GPU VRAM — the more, the better. It's the single most critical factor for running LLMs. For inference tooling, he used Ollama to download and serve the model, though he noted that LM Studio, llama.cpp, and similar tools are all viable alternatives.

From the GPU load screenshots in his test, the entire model fits entirely within the dedicated GPU's VRAM with room to spare for a larger context window, with the GPU running at full load.

The reason VRAM is so critical for LLM inference is that model weights need to be fully loaded into VRAM for efficient computation. For Qwen3.8 (27B parameters), loading with 4-bit quantization requires roughly 14–16GB of VRAM; higher precision requires even more. When a model can't fit entirely in VRAM, the system offloads some layers to system RAM or even disk, causing inference speed to drop dramatically — often from tens of tokens per second down to single digits, making the experience effectively unusable. A MacBook's Unified Memory is shared between CPU and GPU, and while the total capacity can be substantial, Apple Silicon's GPU memory bandwidth is far lower than a dedicated GPU (such as the RTX 4090's ~1TB/s bandwidth), which is the fundamental bottleneck. When selecting or configuring hardware for local inference, GPU VRAM capacity should be the primary consideration — not total system memory.

Three-Tier Difficulty Test: From Simple to Complex

The blogger used his own prompt repository on GitHub, where prompts are ordered from simple to complex. His agent tooling of choice was a minimal and extensible Python framework — he emphasized this lets him interact with the model as directly as possible.

Tier 1: Clock + Stopwatch + Timer

The simplest task was creating an HTML page with a clock, stopwatch, and timer. Given the low difficulty, he skipped planning entirely and asked the model to implement it directly from the description. The file was generated in just over 3 minutes — a solid pace. Opening it in the browser revealed a working clock, stopwatch, and timer, with functional buttons and correct behavior across the board. This tier was an easy pass.

Tier 2: Pomodoro Timer

The second task had hidden complexity: a Pomodoro timer requires multiple states, correct transitions between them, browser notifications, saved analytics and settings, and smooth animations.

For this kind of mid-difficulty task, the blogger would normally have the model draft an implementation plan before executing step by step. This time, he deliberately raised the bar — no planning, just a description, and a single-shot implementation. The result took about 6 minutes to generate (double the first task), but when opened, the design was surprisingly polished, fully functional, and error-free. He said he was "genuinely surprised" that the model pulled it off on the first prompt at this level of quality.

Tier 3: Full-Stack News Aggregator — The Model Hits a Wall

The real challenge was the third task: not just a frontend, but a backend and database too — a full RSS news feed aggregator. To push things even further, the blogger required the project to be written in Nim, a niche programming language he personally enjoys but that hasn't gained mainstream traction.

With this level of complexity, a single prompt was clearly out of the question. His approach: first have the model generate an implementation plan based on the project description, then break the plan into small, discrete tasks.

The Planning Stage Exposed a Clear Weakness

A few minutes later, the plan was ready. At first glance it looked detailed and concrete. But when he fed the project description and the generated plan to OpenAI's newest model (referred to in the video as GPT 5.6) for cross-validation, the cloud model flagged 6 critical issues: the overall concept was sound, but the plan wasn't yet ready for implementation.

This was the first clear limitation he identified — Qwen3.8 is not well-suited for complex architectural design and project planning tasks. Given the model's size, this fell within his expectations.

The Execution Stage Was Impressive

He then switched to his usual workflow: first refining the implementation plan with ChatGPT and breaking it into discrete tasks, then handing those tasks to Qwen one by one for execution. The whole process took about 2 hours, with occasional corrections needed and some help with certain syntax along the way.

The final application ran successfully: the frontend loaded properly, adding an RSS feed pulled in articles correctly, confirming that the backend and database were also working as expected. It's a prototype rather than a production app, but it handles async network operations and database management — and it was built in a niche language. The blogger called it an "excellent result."

Nim is a statically typed, compiled language with Python-like syntax that compiles to near-C performance. Due to its small community and limited public code base, its coverage in LLM training data is minimal — which puts models in a doubly difficult position: they must grasp specific syntax conventions while also navigating knowledge gaps from sparse training data. Using a niche language as a test scenario effectively amplifies a model's generalization ability (and its limits) on uncommon languages, making it a useful way to probe the real ceiling of a coding assistant. That the blogger was able to complete the prototype reflects, in part, the model's ability to transfer general programming logic to Nim syntax — rather than deep mastery of the language itself.

Compared to the Previous Generation: Clear Progress

The blogger specifically compared Qwen3.8 to its predecessor, Qwen3.6. In previous tests, Qwen3.6 struggled with even mid-difficulty tasks, barely managing even when broken into small steps. The new Qwen3.8, by contrast, can handle equivalent tasks comfortably in a single prompt. The improvement is unmistakable.

Conclusion: Where Can You Trust a Local Model?

Across all three tiers, a clear picture of Qwen3.8's capability boundaries emerges:

• Confidently delegate: Single-file utility projects, mid-complexity frontend apps (even single-shot), and well-defined sequential coding tasks.
• Still need a larger model: Complex system architecture design and high-level project planning — these are better handled by a larger cloud model.

The practical workflow he suggests is worth noting: use a large cloud model (like ChatGPT) for planning and solution review, and use the local model for concrete coding execution. The combination protects data privacy while keeping costs in check. He said he'll be upgrading from the previous generation to Qwen3.8. For developers looking to run a local coding assistant and willing to invest in decent VRAM, this model has clearly crossed the threshold of "actually usable."

This "plan in the cloud, execute locally" hybrid workflow strikes a pragmatic balance between privacy and cost. Concrete code implementation often involves internal business logic, proprietary algorithms, or sensitive data structures — all better kept local. Architectural design and solution review, on the other hand, typically deal with abstract descriptions where little sensitive information is actually exposed when sent to a cloud service. Moreover, coding execution tasks consume far more tokens than planning tasks, so offloading that work to a local model can meaningfully reduce API costs. As open-source models continue to improve, the boundary of "what still needs cloud assistance" keeps shifting toward ever more complex territory.