Can a Local LLM Replace Claude Code? SWE-bench Is the Wrong Benchmark

TL;DR: Local LLMs writing a correct function stopped being a problem long ago, but being a reliable Agent — issuing tool calls consistently, keeping consistency across multi-file edits, staying within schema — is what actually decides whether you can close that Claude Code tab. And that, precisely, is what no leaderboard tests.

This is how every "local models replace Claude Code" guide starts: a wall of SWE-bench scores, a triumphant "we're only five points from the frontier," and a GPU shopping list. After running through all of them, here's my conclusion: they're not wrong — they just measure what's easy to measure, not what actually determines whether you can close your Claude Code tab.

Why This Became Feasible All of a Sudden

Two things happened in 2026, and neither of them was "a model got louder on Twitter."

MoE broke the hardware wall. A dense 70B model activates all 70 billion parameters for every token; an MoE model has a huge total parameter count but activates only a tiny fraction per token — Qwen's 35B-A3B activates roughly 3 billion at a time. VRAM is determined by the size of the file loaded, speed by that tiny activated slice. So models that once needed servers now run on gaming GPUs, and after quantization they can even squeeze into a laptop.

Agents learned to speak local. This is the part the 2025 posts generally missed. In January 2026, Ollama added native support for the Anthropic Messages API, with LM Studio and llama.cpp following close behind. That was the unlock: Claude Code no longer needs a proxy shim to talk to localhost — you just point an environment variable at it. The plumbing is no longer the hard part; the conversation can finally move past "can it write code."

MoE (Mixture of Experts) is a sparse-activation model architecture: the model is composed of multiple "expert" sub-networks internally, and each token is routed to only a small subset of them for computation. Its key value is decoupling "total parameter count" from "compute per token" — total parameters determine the model's knowledge capacity, activated parameters determine inference speed and compute cost. This is why a model like Qwen 35B-A3B, with 35 billion total parameters that activates roughly 3 billion at a time, can deliver quality on consumer GPUs far beyond dense models of the same size.

For comparison, a dense model (like Llama 70B) activates all parameters to generate every token, and VRAM and compute requirements scale linearly with parameter count. MoE's VRAM requirements are mainly determined by the total weight volume that must be loaded, while speed is determined by the activated slice. As a result, a quantized MoE model can fit in an 8GB laptop while retaining capability close to a mid-sized dense model. The trade-offs are more complex training, VRAM usage (rather than compute) remaining the bottleneck, and expert load balancing requiring dedicated design.

The Anthropic Messages API is the interface protocol (with fields like system, messages, and tools in the request) that tools like Claude Code use to communicate with models. In 2026, local inference frameworks like Ollama, LM Studio, and llama.cpp implemented this protocol natively, meaning a local model can directly "impersonate" an Anthropic endpoint — Claude Code needs no proxy conversion layer, just pointing requests at localhost via the ANTHROPIC_BASE_URL environment variable. This step eliminated the most troublesome glue layer of the past and is the watershed moment that made local Agent workflows truly usable.

The Number Everyone Quotes — And Why It Misleads You

SWE-bench Verified is the statistic in every headline, and 2026's self-hosted model numbers really are approaching the frontier: a model on a single 24GB card can hit around 77, nearly indistinguishable from the cloud frontier.

But that's exactly where the trap lies:

SWE-bench measures "given a bug, produce the correct patch" — a writing task. It says almost nothing about the four things that actually go wrong when the model is driving your editor.

1. Tool-call validity — Can it reliably (not "usually") produce a well-formed edit_file / run_command call that the toolchain can parse?
2. Long-horizon consistency — Does the 11th call still remember the plan from the 2nd?
3. Context under load — Does quality hold up after stuffing in a real 40K-token repo, or does it collapse well before the advertised window length?
4. Instruction following — You say "don't touch the tests" — does it leave the tests alone?

None of these four are on any leaderboard, yet all of them are why your local setup feels different from the demo.

⚠️ All numbers in this article are vendor-reported. Different guides cite wildly different scores for the same model because they quietly swapped SWE-bench Verified for Pro or Terminal-Bench without saying so. Before buying a GPU, verify on the official SWE-bench leaderboard.

Seven Families Worth Knowing (And What Each Is Actually Good For)

Think in terms of families, not individual checkpoints — every family has multiple sizes, and the right pick is the one that both fits your card and matches your type of work.

Qwen 3.x — The Backbone of Local Coding

If you only run one family, make it this one. Qwen is the dual leader in coding and tool calling among open weights, and that combination is the point. Qwen 3.6 27B (dense) scores roughly 77% SWE-bench Verified, quantizes to about 22GB, runs on a single 24GB GPU or an M-series Mac, and is a reliable tool caller; Qwen 3.5 35B-A3B (MoE) has 35 billion total parameters with roughly 3 billion activated, needs only 8GB VRAM at Q4 quantization, and is the flagship for "frontier on a laptop"; Qwen3-Coder is the code-specialized line, with 30B for daily work and 480B being the strongest open coder if you have 64GB+ unified memory. Best for general coding Agents and tool-use workflows. Weakness: no obvious weakness — it's the safe default.

GLM-4.x — Long-Session Agents on a Budget

GLM punches above its weight at small sizes on long, tool-dense Agent runs. GLM-4.7-Flash scores around 59% SWE-bench and runs on a 5-8GB laptop — this is the usable option for "I only have integrated-GPU-class hardware," not a toy. GLM-4.6/4.x (MIT licensed) is recommended for multi-step Agent sessions, where consistency across multiple calls matters more than single-shot scores. Weakness: lower ceiling on hard single-shot coding than Qwen 27B.

DeepSeek V3.2 / V4-Pro — The Open-Weight Ceiling

DeepSeek^Ad posted the best open-model results — V4-Pro at roughly 80.6% SWE-bench Verified — but it only counts as "local" if you have a server in your desk (80GB+). Best for hardcore coding tasks on server-class hardware or a team's self-hosted box. Weakness: not a consumer-GPU model; this is the upgrade tier, not a daily driver.

Devstral Small 2 — The Permissive-License Workhorse

Around 68% SWE-bench Verified, Apache-2.0, fits in 12-16GB VRAM. It's the best option for "I need a clean license + a mid-range card," and it's a solid Agent coder at its size. Weakness: multi-file reasoning stops below the 27B tier.

Kimi K2.x — Long-Horizon Frontier (If You Have a Rack)

Kimi targets long-horizon Agent work and scores well on the harder Pro benchmark, but running it locally requires heavy quantization or server hardware. Weakness: unrealistic on a single consumer GPU; its scores are based on Pro, so don't compare them directly against the Verified numbers above.

GPT-OSS (20B / 120B) — OpenAI's Open Weights, Codex-Native

Slots naturally into Codex CLI via the --oss flag. 20B runs on consumer hardware, 120B is server-class. Best for teams already standardized on Codex that want zero-friction local fallback. Weakness: Qwen/GLM usually edge it out on open tool-calling reliability.

Gemma 4 — The Lightweight Generalist

Gemma 4 (12B / 27B) is the lightweight-hardware generalist — great when you want all-round ability rather than code specialization on a 16GB machine. Weakness: a generalist rather than a coding specialist; it trails coding-tuned models on hard SWE tasks.

(Honorable mention: MiniMax M2.x — agent-first training and aggressive pricing; worth watching if throughput-per-dollar is your constraint.)

The Efficiency Inflection Point: More Hardware ≠ Better

The chart that should actually drive your GPU decision is: performance vs. VRAM required to reach that performance. The curve rises steeply up to around 24GB, then flattens — the 480B coder actually scores below the 27B on Verified while needing 3x the memory. Lesson: a single 24GB consumer GPU running Qwen 3.6 27B is the self-hosting sweet spot. Beyond that it's diminishing returns unless you specifically need the server-class ceiling.

The Wall the Leaderboards Hide: Tool Calling

This is the reframe that changed how I pick local models: tool calling is a separate skill trained separately from writing code. A model can produce beautiful functions and still fail at the one thing an Agent needs — issuing a valid tool call on the first try, every try.

The evidence converges on a few points: reliability falls off a cliff below about 4B parameters, with sub-7B models emitting malformed calls no matter which Agent wrapper you use; native tool-use training beats bolted-on grammar constraints; the model matters more than the toolchain — swapping Claude Code for OpenCode or Codex won't save a model that mangles JSON; multi-step chains are the death zone — one call is fine, but when five consecutive calls each depend on the last, smaller local models start hallucinating calls, inventing non-existent functions or repeating calls with drifting arguments until the Agent falls into a retry loop.

The practical split: for Agent work, look for 14B-32B models from families with native tool training. In late 2026 that overwhelmingly means Qwen, with GLM-4.x and Devstral Small 2 as credible alternatives. Anything below about 7B is a chat model impersonating an Agent.

Tool calling (function calling) means the model outputs a structured call request (like {name: "edit_file", arguments: {...}}) conforming to a pre-defined JSON Schema for the external toolchain to parse and execute. It's a different training objective from "generating natural language code": the former requires the model to precisely understand the schema's field names, types, and required fields, and to keep arguments consistent with history across multiple turns.

The root of the reliability cliff lies in the training-data distribution: small models see far fewer structured call samples during pretraining than natural language code, and lack dedicated reinforcement-learning alignment. When the task escalates from a single call to chained calls (the Nth call depends on the result of the N-1th), errors compound — a slightly misspelled field name, drifting argument types, or references to non-existent functions all cause the toolchain to reject the call and trigger a retry, and every retry is a full generation, multiplying latency.

The Berkeley Function-Calling Leaderboard (BFCL) is one of the most authoritative tool-calling benchmarks today, scoring across dimensions like "simple functions," "parallel calls," "multi-turn multi-function," and "relevance detection," reflecting model capability in Agent scenarios more directly than SWE-bench. When choosing a model, combine both the SWE-bench and BFCL leaderboards to cover the two independent skills of "writing code correctly" and "using tools correctly".

Setup Really Takes Only 60 Seconds

Because Agents now speak local, this step is utterly unremarkable — and that's the point.

Claude Code → local model (via Ollama):

# 1. Pull a capable coder
ollama pull qwen3-coder:30b
# 2. Point Claude Code at localhost + a throwaway token
export ANTHROPIC_BASE_URL="http://localhost:11434"
export ANTHROPIC_AUTH_TOKEN="local-dummy"
# 3. Run
claude

Codex CLI → local model: Codex talks to any OpenAI-compatible endpoint. With Ollama, the --oss flag points at the local server (:11434); LM Studio serves on :1234/v1. Pin the provider/model in config.toml. Toolchain choice: Ollama is the terminal-first option (there are even one-click launchers for Claude Code / Codex / OpenCode); LM Studio is the GUI. Both work — remember, the model, not the toolchain, determines your tool-calling reliability.

Two gotchas that each cost me an evening: set the context window explicitly — at least 32K, ideally 64K+; the defaults are often tiny and a coding Agent starves instantly. But quality degrades well before the advertised ceiling, and the KV cache eats the VRAM you meant to give to weights — budget for both. Switching back to cloud: unset ANTHROPIC_BASE_URL (or delete it from settings.json). Obvious, until you forget and wonder why "Claude" got dumb.

Hardware Tier Guide

Your Hardware	Pick	Reality
8GB laptop / small GPU	GLM-4.7-Flash, Gemma 4 12B	Usable for single-file assistance; expect retries for Agent work
12-16GB GPU	Devstral Small 2, Qwen3-Coder 14B class	Solid daily coder; clean Apache-2.0 option
24GB GPU (sweet spot)	Qwen 3.6 27B	Worth buying for; reliable tool caller, ~77% SWE-bench
64GB+ unified / multi-GPU	Qwen3-Coder-480B, GPT-OSS 120B	Expert ceiling; diminishing returns
Server 80GB+	DeepSeek V4-Pro, Kimi K2.x	Team box / upgrade tier, not a desktop machine

Where It Actually Breaks (The Honest Part)

After a week on real work, the pattern is consistent: a good local coder gives you roughly 70% of cloud Claude's capability on routine single-file tasks, and the gap widens fast as the work becomes Agentic. Failure modes ordered by how often they bite: schema drift → retry loops (the model emits a slightly malformed edit, the toolchain rejects it, the retry drifts elsewhere, and one hard step can eat a dozen round-trips); multi-file reasoning collapses ("rename in the module and fix the imports" — cloud handles this boringly and reliably, local 30B gets lost by the third file); latency you can feel (a chain that takes seconds in the cloud takes minutes locally — not a tokens/sec gap, but every retry being a full generation); the context cliff is early and steep (the model subtly degrades at around half its nominal window, and "subtly not following the plan" is exactly what you can't afford in an Agent). None of this makes local useless — it makes local a different tool, not a cheaper clone.

Schema drift refers to the phenomenon where a model gradually deviates from a pre-defined JSON Schema across multiple tool calls. Manifestations include: field names being replaced with near-synonyms (file_path written as filename), nested structures being flattened, enum values exceeding the allowed range, required fields being omitted or extraneous fields being added. It's lethal because toolchains typically validate strictly — a single format violation fails the whole chain — and on retry the model often doesn't correct toward the same direction but drifts toward a different error, forming a "retry loop."

The context cliff describes the phenomenon where a model's effective context is far smaller than its nominal window. When input approaches half the nominal window (say 128K), the model's attention weights on early content drop, and it begins forgetting instruction details or design decisions made earlier. This is disastrous for Agents: Agents rely on long-term memory to maintain plan consistency, and once they "forget the goal" they produce plausible-looking operations that deviate from the task. Compressing context, staged summarization, and pinning key constraints at the top of the prompt are common engineering measures to mitigate the context cliff.

So Does It Actually Save Money?

This is the real reason people try, so let's do the math instead of waving our hands and saying "free." Local isn't free — it's prepaid. You trade metered API bills for fixed hardware + electricity + the time spent babysitting retries. A used 24GB GPU that comfortably runs Qwen 3.6 27B: a few hundred dollars, one-time. Electricity under sustained load: real but small, single-digit dollars a month for an individual. For comparison: a heavy Claude Code user can burn hundreds of dollars a month, and a team running Agent clusters sees even more eye-watering bills (the "$4,660 in a month" screenshot is practically a genre now). The honest conclusion: for an individual doing casual coding, local rarely pays for itself versus careful cloud use — the hardware and tinkering costs exceed what you save. What actually flips is high-volume, repetitive, or privacy-bound workloads — especially Agent clusters, where you route the cheap 80% of calls to a local model and spend frontier tokens only on the 20% that needs the frontier. That routing — not "replacing Claude Code" — is the real 2026 win.

Model routing is the core paradigm of 2026's hybrid AI workflows: dynamically dispatching each request to a local or cloud model based on difficulty, privacy requirements, and latency tolerance. A common tiered strategy is "local handles the high-frequency, low-cost, privacy-sensitive 80% of requests (classification, summarization, boilerplate code, single-file rewrites), cloud handles the low-frequency but high-difficulty 20% (multi-file refactoring, long-horizon planning, and tasks where getting the answer right the first time matters)."

The benefits of routing aren't just cost savings: the local part provides zero network latency and the privacy guarantee that data never leaves the machine, while the cloud part preserves the capability ceiling of frontier models. Implementation can be simple heuristic rules (routing by file count, token length, or task type) or training a small classifier to predict "will local fail." The real cost isn't in inference hardware but in building and maintaining the routing logic and fallback mechanisms — which is precisely the fundamental difference between the "replace Claude Code" and "coexist with Claude Code" mindsets.

A Framework for Deciding

Stop asking "which model is best" and ask "which job is this."

Use local when: private or offline (code that can't leave the machine); high-volume, cheap-per-task (thousands of small classification/summarization/boilerplate sub-calls in a cluster); narrow and single-file (one function, one test, one regex, one migration); saving quota (simple 80% local, hard 20% escalated).

Keep cloud Claude/Codex for: multi-file, long-horizon changes where consistency across many steps is the value; large-context work where you need the whole window to genuinely function; "right the first time, fast", where any retry loop costs more than the tokens.

The one line I ended up landing on: local is becoming the default, and frontier models are the upgrade path. That's the opposite of how most teams operate today — and that's where the money is.

The Real Answer

Can a local LLM replace Claude Code? For being your Agent on hard, multi-step, large-context work — not yet, and what's holding it back isn't SWE-bench scores but tool-calling reliability over long sessions. For being the cheap, private, always-available workhorse for the 80% of tasks that don't need a frontier model — absolutely, today, on a single 24GB GPU, and it gets better every month. The mistake is treating it as a binary question. The people actually getting value aren't "replacing" anything — they run both models and know which job belongs to whom.

Numbers in this article are vendor-reported and change monthly. Before spending money on hardware, verify coding scores on the official SWE-bench leaderboard and tool calling on the Berkeley Function-Calling Leaderboard.

One-line Summary

Whether local LLMs can replace Claude Code depends on tool-calling reliability, not SWE-bench scores.

Paragraph Summary

The article argues that leaderboards like SWE-bench only measure "writing the correct patch" — a writing skill — and fail to reflect the four things that actually determine success in Agent scenarios: tool-call validity, long-horizon consistency, context under load, and instruction following. In 2026, MoE architectures broke the hardware wall and local inference frameworks natively support the Anthropic Messages API, making setup a 60-second affair. The author surveys seven families — Qwen, GLM, DeepSeek, Devstral, Kimi, GPT-OSS, Gemma — and points out that tool calling is a training skill independent of writing code, with reliability falling off a cliff below 4B parameters; Agent work should pick 14B-32B models with native tool training. A single 24GB GPU running Qwen 3.6 27B is the sweet spot, with diminishing returns beyond it. Local isn't free but prepaid, and it rarely pays off for individual casual coding; the real win is a hybrid strategy routing the high-frequency 80% of calls locally and the hard 20% to frontier models.