Overseas Engineer Tests Five Open-Source Models in Production: DeepSeek Flash Crowned, GLM Flash Takes the Top Spot

An overseas developer working on the front lines of real engineering recently published a controversial open-weights model ranking. Unlike typical benchmark enthusiasts, his evaluations are based entirely on real-world experience in daily software engineering and multi-agent (agentic) workflows. He makes no effort to hide his growing frustration with frontier models from OpenAI and Anthropic — frankly stating he's "increasingly hating them" — so he decided to focus exclusively on the top 5 from the open-source world.

A Real-World Multi-Agent Workflow

To understand this ranking, you first need to understand how this engineer actually uses these models. His agentic workflow isn't overly complex, but the division of labor is clear: tasks first pass through a "Scout Agent" that collects full context from sources like Confluence, Jira, and Slack, then hands the most relevant pieces to a "Planner."

The Planner needs a "big and juicy" model to generate an execution plan, after which tasks are distributed to one or more "Engineer Agents," each paired with a dedicated Reviewer responsible for feedback loops. All engineers eventually produce a PR, which is then passed to a group of reviewers for a final check.

He's recently been experimenting with assembling four to five independent reviewers into a "council of reviewers," which works exceptionally well for large tasks where the answer isn't clear-cut. The key trick: make sure the reviewers come from different labs. Generate code with one vendor's model, then have another vendor's model review it — this "lab mixing" covers multiple perspectives, and he calls it a tried-and-true method.

A Multi-Agent workflow is an architectural pattern that breaks complex tasks down among multiple specialized AI agents working collaboratively. Unlike having a single model handle everything, each agent is responsible for just one step in the pipeline — gathering information, making plans, writing code, reviewing output — reducing the probability of single-point failures through specialization and feedback loops. The core advantage of this architecture is that different roles have different capability requirements, allowing you to select the model best suited for each stage rather than demanding one omnipotent model to do everything. The Reviewer mechanism is especially critical: it mirrors the human code review process, catching logic gaps or missing context before merging into the main branch — the last line of defense for output quality.

Ranks 5 Through 3: Regulars in Planning and Review

5th Place — Kimi K3: He calls it an excellent model, particularly favoring it for the planning stage. The downside is straightforward — it's too expensive. Even with company reimbursement, he still hesitates to use it frequently. Before DeepSeek^Ad Flash 0731 launched, Kimi was his "daily driver," often paired with Anthropic's Sonnet and Opus. Now it mostly serves as a planner and backup model.

4th Place — DeepSeek Pro: A daily workhorse in his agentic workflow, regularly placed in the planning and review stages. Since the tasks he handles aren't particularly large, token consumption and costs stay manageable, while still providing a unique perspective.

3rd Place — GLM 5.3: He rotates between DeepSeek Pro and GLM 5.3 for planner and reviewer roles. One interesting approach: use GLM when initiating a PR at the ticket stage, then once the PR is out and community feedback comes in, use a different model to do a final review of all the responses. This "multi-perspective self-protection" strategy has been hugely successful for him.

The Crown Moment: DeepSeek Flash and GLM Flash

2nd Place — DeepSeek 401 Flash: This is the model that made him "completely lose it" last Friday. He spent the entire weekend using it, and while it's not ideal as a daily conversational assistant, it's absolutely perfect in the "Engineer" role. He describes it as one of the smartest models he's ever used, with a stunning position on the "cost-performance Pareto curve." Even more remarkably, he says that across multiple benchmarks, 401 Flash outperforms Opus 5 — describing its superiority over previous-generation models as "scorched-earth domination."

1st Place — GLM 5.3 Flash: The top spot comes as no surprise. He admits he once announced he was giving up on this model, only to come crawling back two days later because he liked it too much. The core reason: it's simply outstanding as a conversationalist. When he temporarily switched to DeepSeek 401 Flash for conversation today, he immediately felt something was "off" and didn't like it — switching back to GLM 5.3 Flash restored that familiar feeling.

Of course it has its quirks: it occasionally "leaks context" and tends to break down when utilization exceeds 20%. But as long as there's sufficient oversight, or when used as a primary conversational model, it's cheap, fast, and nearly perfect. He also specifically warns: when calling it via OpenRouter, be careful not to end up on the FP4 quantization version — FP8 is the best balance of speed and cost.

FP4 and FP8 are floating-point precision formats that use 4 bits and 8 bits respectively to represent model weight values. Quantization is the process of compressing a model from high precision (e.g., FP16/BF16) to lower precision, which significantly reduces model size and inference VRAM usage — but accuracy loss worsens as bit depth decreases. FP4 quantization is more aggressive in compression, but the extremely narrow numerical range can lead to noticeable degradation in output quality, especially on tasks requiring nuanced reasoning. FP8 strikes a better balance between compression efficiency and precision retention, and has become the recommended quantization scheme for mainstream inference frameworks like vLLM and TensorRT-LLM. Third-party API platforms like OpenRouter may deploy multiple quantization versions of the same model simultaneously; if users don't explicitly specify, they may be randomly routed to a lower-precision version, causing a noticeable gap between their experience and official benchmarks.

Why Can Flash Models Rival Pro?

The most thought-provoking observation in this ranking is that the Flash series is "taking off" across the board. DeepSeek Flash is extremely close to — and at times surpasses — DeepSeek Pro; GLM 5.3 Flash falls short of GLM 5.3, but the gap is so small it's nearly negligible; even the latest Gemini Flash is closing in on Gemini Pro at a fraction of the cost.

He's heard an unverified rumor that training large models (which he jokingly calls "cooking") sometimes goes wrong — for example, GLM 5.3 is reportedly said to have been "negatively cooked" during training. As for the ultra-low training costs reported by Chinese labs, he remains cautious, viewing the space as subject to a "propaganda war" among vendors and preferring not to over-interpret the numbers.

The more important takeaway is that model progress is far from plateauing. He mentions demos showing that roughly 200GB of lookup tables in the DeepSeek 401 Flash Hugging Face files can be offloaded from GPU VRAM, and that the latest DeepSeek model's KV cache optimizations suggest there's still enormous room for architectural-level improvements. How low can costs go? He's not sure, but one thing is certain — it won't come "at the cost of exterminating humanity" (and he took the opportunity to plug another video of his).

Lookup Tables (LUTs) in neural network inference are an acceleration technique that trades space for time: frequently computed results are pre-stored and retrieved at inference time rather than recomputed, dramatically reducing latency. Offloading roughly 200GB of lookup tables from GPU VRAM to system memory or other storage tiers means the same amount of VRAM can handle larger inference batches, or that deployment can happen on lower-cost hardware. KV Cache (Key-Value Cache) is a technique in the Transformer architecture that reuses intermediate attention mechanism results — in multi-turn conversations or long-text processing, it avoids redundantly recomputing key-value matrices for already-processed tokens, significantly reducing inference latency and compute costs. These two optimizations together suggest that the Flash series' cost advantages don't come purely from making the model "smaller," but from system-level engineering improvements — which is why the performance loss remains minimal.

Final Thoughts

The value of this ranking lies not in the absolute order, but in the fact that it comes from an engineer who has genuinely integrated these models into a production workflow. The DeepSeek and GLM Flash series, with their combination of low cost and high performance, are mounting a real challenge to the dominance of frontier closed-source models. And the practical technique of "lab mixing" — using different labs' models for generation and review — may be more worth borrowing than any individual model's ranking.