Claude Code Model Router Benchmark: 82% Cost Reduction at the Cost of 2 Failed Tests

Most people who use Claude Code tend to lock in a single model for everything — planning migrations, writing tests, summarizing files, polishing READMEs. The problem is that every request gets billed at that model's rate, and on simple tasks, paying for the expensive model doesn't buy you better results.

To quantify this waste, the larebelion team built an open-source project called Rebel Router (MIT license) — a locally running proxy that picks the most suitable model for each individual request. They ran a custom benchmark suite and found: costs dropped 82%, but at the cost of 2 test tasks that Opus would have passed.

What It Actually Is

Rebel Router runs as a local proxy (127.0.0.1:8787) between your coding agent and model providers. Requests from Claude Code (Anthropic Messages format) and OpenCode (OpenAI Chat format) pass through it first, then get dispatched to Anthropic, OpenAI, xAI, Gemini, DeepSeek^Ad, or local Ollama models. All records land in a SQLite ledger for use by a dashboard and CSV export.

The system consists of three components:

• Broker: A model catalog that tracks each model's pricing, speed, and context length, plus quality scores per task category — used to rank the top three candidates for each request.
• Router: The classifier and proxy core, responsible for translating between Anthropic and OpenAI formats (including streaming output and tool calls).
• FinOps: Per-day and per-project budget controls, with per-request cost accounting.

Supported providers include Anthropic, OpenAI, xAI (Grok), Google Gemini, DeepSeek, and local Ollama. You can bring your own API keys for each provider, or use a single OpenCode Zen key to reach GPT, Grok, DeepSeek, GLM, Qwen, and Kimi through a unified endpoint.

FinOps (Financial Operations) is a term from cloud computing that refers to the practice of bringing financial visibility and cost accountability into engineering workflows. In traditional cloud spend management, FinOps emphasizes real-time monitoring, cost attribution to teams and projects, and data-driven trade-offs between performance and expenditure. Rebel Router brings this philosophy to the LLM call layer: each API request is treated as a measurable unit of cloud resource consumption. Through a SQLite ledger, it enables per-day and per-project cost attribution, and uses budget thresholds to trigger automatic model downgrade policies — allowing engineering teams to manage inference costs the same way they manage cloud bills.

How It Decides Which Model to Use

The routing decision follows five steps, and the logic is quite engineered:

Step 1: Classification. Each request is assigned to one of six categories — orchestration, complex_code, bulk_code, unit_tests, qa_automation, or documentation — along with a complexity score from 0 to 1. If a VENICE_API_KEY is configured, it calls Venice's JEV decision model (currently in Beta); if that errors out, returns a nonsensical response, or takes more than 800ms, it falls back to local rules. The rules inspect keywords (supporting Chinese, English, and Spanish), recent tools used (Task, TodoWrite, Edit, etc.), file types (.spec.ts, .md), context size, and Claude Code's request type. Compression and auxiliary calls are always routed to the cheapest documentation-class model.

Step 2: Ranking. The Broker first filters out models below the quality threshold for that category (the bar is raised for complex tasks or opus requests, and lowered for haiku), then weights by quality, price (blended 3:1 input-to-output), and speed — selecting the top three as the primary choice plus two fallbacks.

Step 3: Budget control. At 80% of budget, the router biases toward cheaper models. At 100%, it downgrades to the cheapest acceptable model, or returns HTTP 402 (if onLimit: block is set).

Step 4: Forwarding and logging. For Claude-to-Claude requests, everything passes through unchanged except the model field; for cross-provider routing, full format translation occurs. Each request gets a row in the ledger: timestamp, project, category, provider, model, input/output/cache tokens, cost, latency, and which classifier made the decision and why.

One thing to note: there's no Claude Code hook that can change the session model. The UI always shows the model Claude Code originally requested — the router's status bar is what tells you which model actually responded.

The Venice / JEV decision model mentioned here is a Beta feature on Venice.ai's privacy-first inference platform. JEV (Judgment & Evaluation) is designed specifically for meta-decision tasks — that is, having a small model decide "which model should handle this request" rather than answering the question itself. The core challenge with these "routing classifiers" is latency: if classification takes too long, it erases the cost savings from routing. Rebel Router addresses this with an 800ms timeout circuit breaker — if JEV responds slower than that threshold, it falls back to the local rule engine, ensuring the routing decision itself never becomes a bottleneck. Venice's platform emphasizes that requests aren't logged or used for training, which has some privacy implications for scenarios where code context is sent to the classifier.

Installation and Configuration

Getting started is straightforward: clone the repo, run npm install && npm link to put rebel on your PATH, copy .env and fill in your provider keys, then run rebel start. Providers without keys are skipped; Ollama models are only enabled after you've pulled them locally.

To connect Claude Code, run rebel init claude --global (global) or without --global (current project only). It backs up your settings and writes ANTHROPIC_BASE_URL=http://127.0.0.1:8787, routing the main session, sub-agents, and background calls through the proxy — and injects a project identifier header for per-project budget tracking. The Claude desktop app is an exception; its gateway must be set via Developer → Configure Third-Party Inference.

For OpenCode, run rebel init opencode, then select rebel/auto, or use syntax like rebel/auto:unit_tests to lock a specific task type to a given category. You can also write tags like #rebel:tests, #rebel:docs, or #rebel:plan anywhere in a prompt to have the router use that category for the current session.

The rebel.yaml config file supports excluding providers, pinning specific models to task categories, setting aliases (e.g., letting opus requests pass through directly while haiku routes freely), per-day and per-project budgets, and a localOnly switch to keep a project entirely on-device. Spend monitoring comes with commands like rebel status, rebel spend, rebel export, rebel dashboard, and even rebel classify "<prompt>" to preview how a prompt would be classified.

Benchmark: Let the Numbers Speak

The benchmark consists of 12 real-world tasks, two per category, all scored deterministically — hidden tests, mutation tests, intentionally broken servers, and scoring rubrics, with no model grading another model. Tests were run on October 11, 2026, across 10 models, one attempt per task, total spend of $1.78. Tasks and raw results are publicly available.

A few numbers stand out in the cost vs. pass rate comparison across ten models: gpt-6-luna costs just $0.0010 per task with a 92% pass rate; claude-opus-5-5 costs $0.0799 per task with the same 92% pass rate. In other words, Luna achieved the same 11/12 pass rate as Opus at roughly 1/80th the cost on this task set. The authors note that rows with fewer than 12 tasks are partial runs — don't read too much into a 100% on one or two tasks.

Router vs. Full Opus

The comparison that really matters is smart routing vs. running Opus for everything:

• Cost: Router approach $0.17, full Claude Opus 5.5 $0.96 — 82% cheaper.
• Quality: Router passed 9/12 (75%), Opus passed 11/12 (92%). Of the three failures, two unit test tasks and one QA task were routed to Claude Haiku 5.5 by the router, which produced responses exceeding the 8,000 token limit.
• Latency: Average time per task dropped from 31.7 seconds to 19.5 seconds.
• Classifier: The rule-only classifier (no JEV) correctly categorized 9 out of 12 tasks.

The conclusion is measured: the fact that Luna can match Opus doesn't mean it's a replacement for Opus. What it shows is that a large number of everyday tasks simply don't require the most expensive model.

Mutation testing is a method for measuring the quality of a test suite: it introduces small, deliberate errors into source code ("mutants" — e.g., changing > to >=, or deleting a line of logic) and checks whether existing tests can detect these changes. If a test fails to "kill" a mutant, it reveals a blind spot in coverage. Using mutation testing in this benchmark means scoring isn't just about whether a model can write tests that "look correct" — it's about whether those tests can actually catch subtle logic defects. This also explains why unit test tasks were where the router was most likely to fail: tests written by Haiku might pass syntactically but expose insufficient coverage when mutation testing is applied.

Limitations Worth Taking Seriously

The authors don't shy away from the boundaries of this approach — which is exactly what makes the article trustworthy:

• 12 tasks, one attempt each. Per-category numbers are reference points, not a leaderboard.
• The 82% savings come at the cost of 2 tasks that Opus would have handled fine — and both failures were in testing and QA. For more conservative deployments, use aliases.opus: passthrough to preserve Opus, or classModels to pin specific task types to a fixed model.
• Grok, Gemini Pro, Ollama, and GPT models other than Luna were not tested; JEV wasn't tested either — the benchmark used the rule-based classifier.
• Anthropic officially doesn't support routing Claude Code to non-Claude models. It works, but translation may drop Claude-specific features.
• Both JEV and Venice's Decisions API are in Beta; the interface may change.
• Costs are calculated using official catalog prices (token count × unit price).

The authors close with a pointed reminder: the number you should really be watching is that 75%. A router that saves you 82% is only worth it if the tests it skips aren't the ones that should have been alerting you. So before fully trusting your test suite to it, keep a close eye on the ledger for the first few weeks.

Closing Thoughts

Rebel Router's value isn't that some model is cheap — it's that it makes "model selection," a long-overlooked decision, explicit and measurable. For teams that use Claude Code as a daily productivity tool, the idea of "per-request model assignment + FinOps ledger" may be more pragmatic than blindly upgrading to the most powerful model available. But it's also honest with you: cheap has a cost, testing and QA are where things are most likely to go wrong, and you shouldn't blindly trust automatic routing.