Open-Weight Models Are Falling Behind: Three Structural Reasons

A longtime open-source advocate now argues open-weight models can no longer catch up to closed-source frontier models — and the gap will widen structurally.
The author was once a staunch supporter of open-weight models, citing DeepSeek V3 Flash and R1 catching up to closed-source flagships. Now he's flipped to bearish for three reasons: closed-source labs are acquiring compute exponentially faster; distillation shortcuts have been largely blocked by Anthropic via degraded reasoning traces and API blocks; and recursive self-improvement only happens inside closed-source labs, with research capabilities deliberately restricted from being distilled. He also cites T3 code production data showing open-source models account for just 0.5% of request turns, and uses the MaxBench experiment to expose serious benchmark gaming. He lists variables that could prove him wrong, but until then he no longer includes open-weight models in his real workflows.
A developer who has long championed open-source technology recently said in a video what he least wanted to admit: open-weight models have fallen behind closed-source frontier models, and that gap will never close again. This isn't emotional doom-saying — it's a conclusion drawn from his own platform usage data, public benchmarks, and observations of lab behavior.



The closest they ever came
The author recalls that when DeepSeekAd V3 Flash appeared, he first realized he could download a model locally that was nearly on par with closed-source flagships. What shocked him even more was DeepSeek R1 — shortly after OpenAI's O1, when neither Anthropic nor Google had a reasoning model yet, R1 not only caught up to O1 but surpassed it on many tasks. He describes that period as one where the phrase that kept coming up in conversations with another YouTuber was "fast, cheap, and inexplicably good."
The author admits that in early 2025 he briefly believed open-weight models had fully caught up with closed-source. But now his judgment has reversed: "I've completely stopped betting on good open-weight models, because I don't think they can reach the level needed today." He gives three reasons — compute scale, distillation, and recursive self-improvement — which together form an insurmountable structural barrier for open-source models.
Reason one: the compute gap is widening exponentially
The author uses a deliberately simplified set of numbers to illustrate the trend: closed-source labs may double or even quadruple their GPU counts each year, while open-source labs subject to import restrictions can only manage roughly 1.5x to 2x. The absolute numbers don't matter — what matters is the slope of the curve: this is an "exponential gap."
He acknowledges that Chinese labs and independent researchers have pulled off many clever engineering innovations by cobbling together old GPUs and expanding VRAM, but the scaling law itself hasn't been falsified: more compute usually means better models, and the industry still hasn't found the point where this curve breaks down. Closed-source labs are acquiring more GPUs significantly faster than the open-source camp, and this gap will only continue to widen.
The Scaling Law is the core context for understanding this gap. It refers to the fact that, given simultaneous increases in data volume and parameter count, model performance (typically measured by cross-entropy loss) improves predictably as a power law with training compute. OpenAI first systematically described this phenomenon in a 2020 paper, and since then the practices behind models like GPT-4 and Gemini have confirmed that the curve shows no clear saturation across a fairly wide range. This is precisely why the industry has formed an almost engineering-faith consensus that "more GPUs = stronger models." When the author says "the point where the curve breaks down hasn't been found," he means no one has experimentally proven that scaling has stopped working — and until that break point is demonstrated, differences in the slope of compute investment will translate directly into differences in capability.
Reason two: the distillation shortcut has been blocked
Distillation used to be the main method for open-source models to catch up: using closed-source model outputs to supplement training data they couldn't otherwise obtain. The author believes Anthropic has essentially won this "war on distillation." He cites his own observation — Opus's reasoning traces are so poor in quality that he suspects it's deliberate, designed to prevent third parties from effectively using that data for training.
He references a report from Orca Router indicating that attacks on AI inference infrastructure are shifting toward stealing credentials and hijacking inference capacity. Many assume the black market is just reselling tokens at a markup for arbitrage, but the author believes the real payoff is data: these relay services log every request and response, then sell them in bulk to labs that need training data. "Distillation matters so much to these labs that they're willing to commit espionage and global cybercrime for it."
Another piece of evidence: a developer found that Kimi K3.1 Pro and GPT-6 Astra produce nearly identical results on UI generation tasks, while other models each have their own style. The author quotes original poster Dara's comment: "OpenAI models are so bad at frontend that they've ironically become an effective way to detect distillation."
Knowledge Distillation was originally proposed by Hinton et al. in 2015. The core idea is to have a student model learn the "soft label" distribution output by a teacher model rather than just hard labels, thereby transferring the capabilities of a large model to a smaller one. In the open-source community context, distillation has been extended to mean: using closed-source model APIs to batch-generate high-quality Q&A, code, or reasoning chains, then using that data as supervised fine-tuning corpora. This approach costs far less than pretraining from scratch and has been an important lever for open-source models to catch up quickly. But it depends on one precondition — that the teacher model's outputs can be obtained legally, reliably, and at high quality. Once closed-source vendors tighten their terms of service, output quality, and API layers simultaneously, the cost-effectiveness of this path drops sharply.
Reason three: recursive self-improvement only happens on the closed-source side
The author cites a report from Meter about Anthropic (much of the evidence in which was redacted), stating that as of October 2026, the rate of capability growth on well-defined tasks may be roughly double what it was between March 2024 and October 2025 — somewhere in the 1.3x to 3x range.
What he keeps coming back to is the productivity improvement curve self-reported by Anthropic employees: from 20% to 25% in the 3.7 Sonnet era, to 2x to 2.5x with Opus 4.5, to the 5x range with Mythos 5, while the internal codename Model 4's self-reported figure is 12x. This is a log-scale chart — on a linear scale the curve would be even steeper.
The author's key argument: Anthropic doesn't prevent its models from being good at AI research during training, but it suppresses these behaviors before public release and hard-blocks them at the API level. This means that even if open-source labs wanted to distill these capabilities, they couldn't get usable data. Capabilities like writing code, modifying frontends, and porting applications can be distilled — but the ability to do AI research itself cannot.
Recursive Self-Improvement refers to the process of a model participating in improving itself or the next generation of models — for example, automatically generating training data, proposing architectural changes, writing training code, and evaluating experimental results. It's seen as a key variable because once a closed loop forms, capability growth no longer depends linearly on the number of human researchers but accelerates with the model's own capabilities. This is also why the author emphasizes that "AI research capability cannot be distilled": the ability to write frontend code can be imitated from outputs, but meta-capabilities like "how to design experiments and how to judge whether a training change is effective" tend to live in internal toolchains, evaluation pipelines, and researchers' tacit knowledge — they're very hard to fully extract through API outputs.
Usage data: almost no one uses open-source models in real work
The author shares statistics from his own product, T3 code: open-weight models account for 0.5% of total request turns, 0.7% of total tokens, and the share of installations that have used an open-source model at least once has dropped from 16% to 6%. He acknowledges that the T3 code sample isn't large, but emphasizes that its daily token throughput exceeds that of OpenRouter.
In his view, the high traffic for open-source models on OpenRouter comes mainly from free tiers rather than real production use. He infers from this that the only survival path for open-source labs is to pump out impressive benchmark scores, secure funding, then pump out the next benchmark — a cycle.
Benchmarks are no longer trustworthy
The author uses an even sharper experiment to make the point. DataCurve open-sourced a code benchmark called DeepSWE, and developer Justin Lowe made minor modifications to its 89 public tasks to construct MaxBench — the same problems with slightly different requirements. The result: Opus 5.5 was the only model whose score didn't drop; GLM 5.3 lost 12 points, DeepSeek V4.1 Flash lost nearly 30 points, and DeepSeek V4 Flash's score was cut in half.
He also tells his own SkateBench story: an open-weight model suddenly achieved a perfect score, and after he doubled the number of problems, it didn't get a single new one right.
Benchmark contamination and overfitting are long-standing problems in the evaluation field. Once a public benchmark is released, its problems can enter the training corpora of subsequent models, inflating scores; model developers may also optimize specifically for a benchmark's problem types, formats, and scoring scripts. The reason the MaxBench experiment the author describes is so convincing is that it makes only "minor modifications" to the original problems, preserving the difficulty distribution while breaking the model's memory match to the original problems — thereby revealing how much of a score comes from generalization versus memorization. This also explains why a high score on a single benchmark is increasingly difficult to use as evidence of capability, and needs to be cross-validated with perturbation tests, private holdout sets, and real production telemetry.
Conclusion: he hopes he's wrong
The author closes with a "natural experiment": Anthropic and OpenAI are similar on other dimensions, but the former has models capable of improving models while the latter doesn't — and the result is a clear gap between the two companies. He believes this already previews the future of open-source versus closed-source. He also lists the variables that could prove him wrong: relaxed export controls, new distillation methods, new training paradigms. "More than any video I've made recently, I really hope everything I've said is wrong." But until proven otherwise, his choice is to stop including open-weight models in his real workflows.
Related articles

Robinhood's Hybrid AI Reviewer: 60% of Data PRs Auto-Merge
Robinhood's data platform team shares a hybrid AI PR reviewer: deterministic scripts verify facts, AI Agents judge contextual risk, and humans keep key decisions. About 60% of data pipeline PRs merge without human review, cutting review workload ~75%, saving ~132 engineering hours monthly, and reducing related incidents ~30%.

Making Marketing Videos with Claude Code Is More Efficient Than AI Video Generation
Generate marketing videos programmatically with Claude Code—more flexible and cheaper than AI video generators. Reference a style, generate a storyboard, output content as code, and make edits instantly.

Is AI Becoming a Climate Disaster? A Deep Dive into the Carbon Cost of Compute Expansion
AI compute expansion is driving dual energy pressure from training and inference, while data centers face challenges in power, cooling, and water. This article examines AI's climate impact and possible sustainable paths for the industry.