Ornith 9B/35B Hands-On: A New Local Deployment Option for Code Agents

A New Open-Source Model Built for Code Agents

In the race for locally deployable large models, a new contender is drawing attention. The Ornith 1.0 series, trained and open-sourced by the Deep Reinforce team, is purpose-built for code agents and real-world software development tasks. The two most suitable configurations for local deployment are Ornith 9B and Ornith 35B MoE.

Ornith isn't built from scratch — it's the result of deep, targeted reinforcement on top of Qwen 3.5. While Qwen 3.5 is a well-balanced general-purpose model, Ornith layers on substantial improvements in code repository understanding, tool calling, multi-file editing, and automated bug fixing — precisely the capabilities that matter most in code agent scenarios.

According to public benchmark data, Ornith 9B scores 69.4 on SWE-Bench, approaching Qwen 3.5-35B's score of 70. Ornith 35B is even more impressive, surpassing Qwen 3.5 and even edging out the newer Qwen 3.6 — earning the authors' claim that it is "the strongest 35B MoE model currently available."

SWE-Bench (Software Engineering Benchmark) is a benchmark developed by Princeton University to evaluate code agent capabilities. It draws tasks from real GitHub issues, requiring models to read problem descriptions, understand repository structure, generate patches, and pass unit tests — with performance measured by "successful fix rate." Unlike traditional code completion exercises, SWE-Bench tasks closely mirror real engineering work, involving cross-file modifications, dependency understanding, and bug localization. It is widely regarded as one of the gold standards for evaluating code agent performance in practice.

It's also worth clarifying the 35B MoE architecture. MoE (Mixture of Experts) is a sparsely activated neural network design. Traditional dense models activate all parameters during every inference pass, while MoE models partition parameters into multiple "expert" sub-networks, with only a few experts activated per inference (selected dynamically by a router network). This means that while the Ornith 35B MoE has 35 billion total parameters, far fewer are actually activated per inference — giving it a high capability ceiling while keeping inference speed and VRAM requirements closer to a much smaller dense model.

Not Trained into a "One-Trick Pony"

Community fine-tuned models often suffer from a common problem: coding ability improves after specialized training, but Chinese language quality, general reasoning, and even image recognition noticeably regress — essentially "breaking" the model in other areas.

Ornith's performance in our hands-on testing is notably different. Code and agent capabilities are clearly enhanced, while Chinese writing fluency and visual understanding remain at a high level. This ability to retain foundational skills is especially important for local deployment, where a single model often needs to handle diverse tasks rather than a single narrow function.

Setting Up a Local Deployment Environment

Starting from the official Hugging Face repository, here's how to set up a local inference service on Linux using llama.cpp — with a focus on a few common pitfalls.

Obtaining Model Weights and the Image Decoder

The official repository provides GGUF-quantized versions across multiple quantization levels. All tests here used Q6_K quantization. GGUF (GPT-Generated Unified Format) is a model file format introduced by the llama.cpp project, optimized for local inference and supporting multiple quantization levels to suit different VRAM configurations. Q6_K uses 6-bit quantization combined with K-quants grouping, compressing model size while preserving as much precision as possible — offering less degradation than Q4 quantization, and a smaller footprint than Q8, making it generally a solid balance between quality and VRAM usage.

One important note: the official repository does not include an mmproj file (image decoder) for visual tasks. The mmproj file is the "projection layer" weight that bridges the vision encoder and the language model in multimodal architectures. Images are first converted into feature vectors by a standalone vision encoder, then passed through the mmproj to align dimensions before entering the language model backbone. Since Ornith is based on Qwen 3.5 and its visual processing module is unchanged, the visual feature space is fully compatible — meaning the Qwen 3.5-9B mmproj file can be reused directly, and it works well in practice.

Compilation and Startup

The test machine is equipped with an NVIDIA V100 GPU. When compiling llama.cpp, CUDA acceleration must be enabled in the build configuration. Once compiled, the latest executable environment is ready to use.

Startup commands differ between single-GPU and multi-GPU modes: by default (no prefix), all available GPUs are used; a parameter can be passed to restrict to a single card. In testing, the 9B model runs smoothly on a single V100; the 35B MoE requires two GPUs — on a single card, inference speed drops to an impractical level.

Key parameters: -m specifies the model file to load; --mmproj specifies the projection file needed for multimodal image understanding (omit this for text-only tasks to save VRAM); the remaining parameters configure the listening address and context size.

A Rule of Thumb for VRAM Allocation

The most common pitfall in local deployment is ignoring the balance between model size, VRAM capacity, and context length. This requires understanding the role of the KV cache (Key-Value Cache): a critical mechanism in Transformer architectures that accelerates autoregressive generation by storing previously computed key-value pairs in VRAM, avoiding redundant computation for each new token. KV cache VRAM usage scales proportionally with context length — a 128K context window can require several gigabytes of KV cache, potentially exceeding the model weights themselves.

This means model weights must never fill VRAM entirely; sufficient headroom must be reserved for the KV cache and the inference process. For example, a model that appears to just fit within 16GB of VRAM may immediately run out of memory once connected to an agent with a 128K context requirement.

Three Rounds of Baseline Capability Testing

To verify whether Ornith retains Qwen 3.5's foundational capabilities after code-focused reinforcement, three test rounds were designed.

Round 1: Chinese Writing

The task: write a campaign speech for a kitten running for election. Ornith 9B generated output quickly on a single V100, and the text quality was quite good. Compared to many fine-tuned models that suffer from degraded Chinese fluency — often producing English-mixed or Traditional Chinese output — Ornith's Chinese expression remained solid. The 35B MoE's Chinese output was on par with the 9B.

Round 2: Complex Logical Reasoning

This round used the "Hardest Logic Puzzle Ever" — a multi-layered problem with several traps: you don't know which alien words mean "yes" and "no"; you don't know whether the entity you're questioning is the truth-teller, the liar, or the random responder; the random responder answers with no pattern; and you can only ask one question at a time, with just three questions total.

The puzzle was formalized by logician George Boolos in 1996 and is considered the ultimate variant of classic logic puzzles. The solution requires constructing counterfactual conditionals to work around the random responder's unpredictability, and using doubly-nested questions (asking how a god would respond if asked another question) to eliminate the unknown yes/no vocabulary. The puzzle tests not just propositional logic, but also strategic design and optimal decision-making under information constraints — making it a landmark benchmark for evaluating advanced AI reasoning.

Ornith 35B MoE produced the correct answer in approximately 30 seconds, demonstrating strong reliability. The 9B model did not pass, with clear logical errors in its reasoning. Notably, the original Qwen 3.5-9B also failed this puzzle — indicating that such complex reasoning tasks simply exceed what a 9B parameter model can reliably handle, rather than representing any regression from Ornith.

Round 3: Invoice Recognition

The third round tested visual understanding by asking both models to extract information from an electronic invoice containing 28 fields. This is a classic real-world local deployment scenario — sensitive financial data cannot be uploaded to the cloud, so the local model must have sufficiently reliable OCR and document understanding capabilities.

This task goes well beyond simply "reading text." The model must understand the invoice layout, correctly distinguish item names, quantities, amounts, and tax figures within table structures, recognize negative values from discount or reversal lines, and handle long product names, special symbols, and Chinese uppercase monetary amounts.

After running a Python script to perform one-click image extraction and automated comparison, the results showed: both the 9B and 35B models successfully extracted all 28 fields with a 100% match rate. This confirms the consistently reliable visual understanding of the Qwen family — and Ornith, as a derivative model, lived up to that reputation.

Summary and Outlook

Across all three test rounds, the Ornith series delivered a satisfying performance: substantially enhanced code and agent capabilities, while successfully retaining Chinese language fluency, complex reasoning (35B), and multimodal image understanding. This "well-rounded" characteristic makes it a serious contender in the local deployment space.

That said, this was only a baseline capability evaluation. What truly determines a code model's value is its real-world performance on actual development tasks when integrated with an agent framework, and how it stacks up against competitors like Qwen 3.6 and GLM-4 in head-to-head comparisons. According to the authors, a more comprehensive comparative evaluation is planned for upcoming content. For developers still wrestling with the question of "which model should I actually use for cost-effective local deployment," Ornith offers at least one new and highly competitive answer.

Key Takeaways