Ornith 1.0 9B Model Local Test: Tower Defense Game Exposes Small Model Coding Limitations

An Open Model Built for Agentic Coding

San Francisco-based company Deep Reinforce recently released the Ornith 1.0 open model series, designed specifically for agentic coding. Agentic coding means an AI model doesn't just generate code snippets — it autonomously plans multi-step tasks, calls tools, reads the filesystem, executes terminal commands, and iteratively refines its work based on feedback, much like a human developer. The key distinction from traditional "code completion" lies in autonomy and closed-loop capability: the model handles the entire pipeline from understanding requirements to delivering a working product, without requiring step-by-step human intervention.

The series comes in four sizes: a 9B dense model, a 31B dense model, a 35B MoE model, and a 397B MoE model. It's worth clarifying the two architectures: Dense Models activate all parameters on every inference pass, with compute proportional to parameter count; MoE (Mixture of Experts) models use a gating mechanism to activate only a subset of "expert" sub-networks at a time, allowing massive total parameter counts while keeping actual inference costs far lower than a dense model of equivalent size — the 397B MoE may carry roughly the same compute cost as a dense model of just a few dozen billion parameters.

The 9B, 35B, and 397B versions are all open weight, meaning users can download and run them locally; the 31B version has not yet been released. It's worth noting that "open weight" and "open source" are distinct concepts in the industry — open weight simply means the model parameter files are publicly downloadable, but training data, training code, and commercial usage terms may still be restricted. Users should carefully review the license terms before any commercial deployment.

According to the release paper, Ornith 1.0 is built on top of pretrained Gemma 4 and Qwen 3.5 — the former is considered one of the strongest models for agentic task completion, and the latter is a well-known local AI coding model. This "standing on the shoulders of giants" approach made Ornith 1.0 noteworthy from the moment of its release.

This article focuses on the smallest 9B dense model, testing its real-world coding ability on an M4 Mac mini with only 16GB of RAM.

Benchmark Credibility and Its Limits

The paper includes three performance charts, one each for the 397B, 35B MoE, and 9B models. The vendor claims the 35B MoE model outperforms Qwen 3.6 35B — a bold claim.

But any pragmatic developer knows: the most reliable benchmark is always running your own real use cases on your own hardware. Official numbers rarely translate directly into day-to-day development experience.

There's a critical distinction worth highlighting. Mainstream benchmarks like Terminal Bench 2.1 and SWE Bench Verified/Pro do not measure the ability to build complete applications from scratch. SWE-Bench, proposed by Princeton University, tasks models with locating and fixing bugs in real open-source GitHub repositories based on issue descriptions, verified against unit tests. Terminal Bench focuses on multi-step task completion in terminal environments. Both test "repair" rather than "creation," covering scenarios such as:

• Running within an existing harness: Can the model correctly execute tool calls and complete terminal tasks?
• Fixing bugs in unfamiliar codebases: Given a never-before-seen codebase and a bug, can the model read, locate, and fix it?

Both capabilities are valuable, but they're fundamentally different from "writing a complete program from scratch" — which is precisely what most independent developers want most, and the core goal of this test.

Local Deployment Test on a 16GB Mac mini

Test hardware: M4 Mac mini, 16GB RAM. With roughly 4GB reserved for the OS and screen recording, the model had approximately 12GB available. The free desktop app LM Studio was used as the runtime.

Ornith 9B in LM Studio is available in both MLX and GGUF formats. MLX is Apple's machine learning framework designed specifically for Apple Silicon (M-series chips), leveraging the Unified Memory Architecture (UMA) and Neural Engine (ANE), with particular strengths during the prefill stage. GGUF (GPT-Generated Unified Format) is the general-purpose quantized format from the llama.cpp ecosystem, offering broader cross-platform compatibility for Windows, Linux, and heterogeneous environments. This test used the more universally compatible GGUF format.

Context length performance was impressive: model weights occupy roughly 6.2GB, and within a 12GB memory budget, the model supports approximately 186,000 tokens of context — compared to roughly 70,000 tokens achievable under the same memory constraint when testing Gemma 4 12B. This gap is primarily due to differences in KV cache memory consumption. The KV cache (Key-Value Cache) is a core optimization in Transformer inference — it caches previously computed attention matrices to avoid redundant computation during autoregressive generation, but at the cost of memory that grows linearly with context length. With Flash Attention enabled (an algorithmic optimization that reduces memory I/O by reordering computations) and the KV cache quantized to Q8, the full context window can be supported with as little as 10.29GB of memory usage.

Generation speed in testing was approximately 16 tokens/sec (the day before it reached roughly 18 tokens/sec during benchmarking; the difference is likely due to other applications consuming memory and causing throttling). For short messages, MLX and GGUF perform similarly, but MLX's advantage becomes more pronounced as context grows.

Tower Defense Showdown: 9B vs 35B

To test real-world agentic coding performance, the task was: build a tower defense game from scratch in a single HTML file. The same task was previously attempted with Qwen 3.6 35B, which produced a working result on the first try in under four minutes.

Development environment: VS Code + the PyAgent agentic framework. The author acknowledges a preference for Claude Code, but on a small model + low-memory device, Claude Code's wait times were too long — PyAgent was faster and more practical.

For comparison, the Ornith 35B (4-bit MLX) model was invoked remotely from another Mac Studio via LM Studio's LM Link feature. The speed difference between the two devices was stark: the 9B ran at ~16 tokens/sec, while the 35B reached ~100 tokens/sec — five times faster. That said, this is not a strict apples-to-apples hardware comparison.

9B Results: It Runs, But It's Unplayable

The 9B model generated roughly 900 lines of HTML — comparable in length to the 35B's 750 lines. But opening it in the browser was disappointing — the game was essentially non-interactive: towers didn't shoot, and enemies didn't move properly. Attempts to have the 9B debug its own code led to an endless loop of changes that made things progressively worse.

35B Results: Iterations Lead to a Playable Game

The 35B model performed significantly better. There was a minor hiccup during recording (the same prompt had produced a working result on the first try the day before), but after explaining that "towers aren't shooting" and going through a few rounds of iteration, the model delivered a genuinely playable tower defense game: archers, cannons, frost mages, and snipers all fired correctly, gold was consumed, and waves progressed. Visually simple, but logically complete and mechanically sound.

Core Finding: The "Precision" Bottleneck of Small Models

This comparison test clearly reveals the fundamental shortcomings of small-parameter models in code generation scenarios. The author's summary is direct:

When using a 9B model for coding tasks, either lower your expectations or narrow your scope.

The 35B model has the capacity to produce high-quality work, while the 9B model's problem isn't that it isn't "smart enough" — its reasoning actually reads quite coherently — it's a lack of precision, manifesting as:

• Creating functions but forgetting to declare them;
• Generating empty function bodies;
• Corrupted tool call formats that PyAgent cannot parse;
• Misspelled variable names and functions defined but never called.

This phenomenon has theoretical backing in Scaling Law research: larger parameter counts allow the model to simultaneously track more variable names, function signatures, and call relationships, reducing the probability of "locally correct, globally inconsistent" errors during long code generation. The error rate in code generation tasks follows a power-law relationship with parameter count — this is precisely why the "glue work" of stitching an application together is so hard for 9B-class local models. This is not unique to Ornith — other models at similar scales like Qwen face the same challenges — it's a universal limitation of parameter scale.

Final Thoughts

Ornith 1.0 is a genuinely well-intentioned open model release: the 186K token extended context, friendly MLX support for Apple Silicon, and the technical foundation built on Gemma 4 and Qwen 3.5 are all commendable.

But for users hoping to "generate a complete app in one prompt" on lightweight local hardware, the 9B model still can't handle building from scratch. Its sweet spot is executing well-scoped small tasks within an existing framework, or handling limited-scope work with careful prompt engineering. Developers who want truly usable output still need models at the 31B scale or above.

The boundaries of local LLMs are expanding rapidly — but the principle that "more parameters buy you more precision" remains valid for the foreseeable future.

Key Takeaways