[KongchangAI]
· 4 min read· 2,023 words

Jonathan Blow: The AI Coding Myth and the Truth About Software Quality

Jonathan Blow: The AI Coding Myth and the Truth About Software Quality

Blow argues AI coding lacks judgment, tenfold productivity is unproven, and the industry's experience gap is the deep root of quality problems.

Jonathan Blow systematically criticizes the software industry's chronic problems and the boundaries of AI coding. He notes that software engineering never established real professional accountability, methodological trends cycle without substantive quality improvement, rapid expansion dilutes experience and breaks knowledge transfer, and ever-growing complexity keeps quality stagnant. He sees current LLMs as efficient search algorithms lacking judgment, whose redundant output accumulates as technical debt, with no evidence for tenfold productivity. The only practical gain is AI-driven vulnerability scanning. He warns the AI startup wave shows bubble traits and advocates knowledge guardianship and regularly paying down technical debt.

Why Does More AI-Generated Code Make Software Worse?

Legendary game developer Jonathan Blow, creator of Braid and The Witness, raised a sharp observation in a deep-dive interview: if AI were truly as good at writing code as everyone claims, the companies building such products should be the most valuable companies in the world. But in reality, he looks around and software is still the same—crashing when it should crash, full of bugs where bugs exist.

The conversation centered on the chronic problems of software engineering, the real boundaries of AI coding, and why the entire industry is seeing no commensurate output despite exploding investment—a rare candid discussion.

Software Engineering Has Never Established Real Professional Accountability

Blow aimed his criticism at the software industry's long-standing lack of accountability culture. In civil engineering, if a bridge collapses, engineers are held responsible—historically, there was even a tradition of "living under the bridge first." But in commercial software, it's been a wilderness for decades. Data breaches, performance issues, a single web button that can't produce a predictable result—these failures are so common that everyone assumes "software is just like this."

He was equally blunt about the "arrogance" of various methodologies: in the '80s, not doing object-oriented programming meant you were a bad engineer; in the '90s, it switched to functional programming, cycling every decade, and Rust evangelists are the new batch. The excitement of these paradigm debates is wildly disconnected from actual engineering quality.

Interview screenshot

One of the key differences between software engineering and traditional engineering disciplines like civil engineering lies in professional accountability mechanisms. Traditional engineering fields typically have licensing systems, industry standards, and legal liability frameworks: engineers may face license revocation, civil damages, or even criminal liability if design flaws cause accidents. The software industry lacks equivalent constraints—there is no unified professional certification, and data breaches or system crashes caused by code defects are often accepted as "industry normal," rarely traced back to specific individuals. This structural difference externalizes the cost of software quality problems to users and society rather than having developers bear them. The "wilderness" state Blow criticizes refers precisely to this industry ecosystem lacking mature professional ethics and responsibility frameworks.

The Experience Gap: Rapid Growth Dilutes the Industry's Collective Judgment

Why are old systems more stable? Blow offers an intuitive arithmetic: the software industry has expanded rapidly for a long time. If the number of people keeps doubling, then half of all practitioners have zero years of experience, a quarter have only one year, and the average experience converges to a very low number. His own experience confirms this—he studied computer science at Berkeley for four or five years, considered himself one of the best students, but upon graduating and actually doing hard things, he realized "most of it I simply couldn't do," and it took about five or six years of work before he felt he'd leveled up.

This means that when no one in the industry has accumulated six years of experience, no one really knows how things should be done. The "person who's been here five years and remembers why we made this decision" is increasingly rare in companies, knowledge transfer breaks down, and what remains is layer upon layer stacked on a massive technical garbage heap.

Complexity Only Grows, and It Gets Harder to Untangle

Blow uses a node graph to describe a program: nodes are code, edges are the implicit constraints and consequences between pieces of code. A good programmer's job is to maintain this increasingly dense graph in their head, understanding the cascading effects of each decision. And software has a characteristic—it hardens over time. The larger the system, the more any change can affect the whole body; so-called "decoupling" is fundamentally impossible at the limit.

He gives two examples. One is Unix's "everything is a file" design: the original intent was simplicity and speed, but today it's so complex it's unrecognizable, yet due to lock-in effects, no one has the incentive to tear it down and start over. The other is systemd, which ballooned from a simple init script to over 1.5 million lines of code, swallowing login, messaging, and everything else. His assessment is direct: this is a typical sample of the entire industry's problems.

Interview screenshot

Systemd is a core component in Linux systems for initializing and managing system services. Its original design goal was to replace traditional SysVinit scripts, providing faster boot times and more unified process management. However, as versions iterated, systemd continuously expanded its functional boundaries, gradually taking over log management, network configuration, login sessions, scheduled tasks, and other areas originally handled by independent tools, with its codebase ballooning to over 1.5 million lines. This "feature creep" is a typical phenomenon in software engineering: a component born to solve a specific problem, lacking boundary constraints and alternative competition, continuously devours surrounding functionality and eventually evolves into an unwieldy behemoth that is difficult to replace. Blow sees it as a microcosm of the industry's systemic problems—once complexity accumulates to a critical point, the cost of refactoring far exceeds the cost of tolerating it.

AI Is a Powerful Search Algorithm, Not an Engineer That Reasons

When it comes to AI coding, Blow's attitude is not wholesale rejection but precise demarcation. He believes current large models are essentially systems that are "particularly good at understanding what you want and then doing large-scale search"—fast and cheap, but lacking in creativity and judgment. He mentions that when asked to explain its reasoning, AI will produce a narrative like "I weighed this because...", but looking at the output, it's clear that no good trade-off actually occurred.

He uses a personal experience to illustrate AI's real value range: he needed to color table cells, used AI to generate about thirty lines of Visual Basic code, and it worked in one shot, with good results. Learning from scratch might have taken five hours; it actually took about one hour. But the key premise is—this code is disposable, not entering any system that requires robustness.

Once complexity rises, problems emerge. AI produces far more code than an excellent human programmer for the same problem, and the excess is all technical debt to be paid in the next round. The optimistic argument of "letting AI pay the debt" doesn't account for error rates accumulating with scale, Blow believes, and people who rely purely on vibe coding will quickly hit a ceiling.

Vibe coding is a programming approach that became popular in recent years with the proliferation of large language models. It refers to developers not deeply understanding code logic, merely describing requirements in natural language, having AI generate code and running it directly, advancing development based on "feel" rather than rigorous reasoning. This approach is extremely efficient for prototyping and one-off scripts, but lacks systematic consideration of edge cases, error handling, and system constraints. As project scale grows or long-term maintenance is needed, unreviewed AI-generated code introduces numerous hidden defects, and since the developer doesn't truly understand the code, debugging and fixing costs are actually higher. The "hitting the ceiling" Blow warns about refers precisely to the reliability bottleneck that vibe coding inevitably encounters as complexity rises.

Where Is the Evidence for the Productivity Explosion?

This is the sharpest question of the entire conversation. If AI truly brought tenfold productivity and has been doing so for several years, we should see the equivalent of decades of software progress. But Blow's conclusion as an outside observer is: almost zero. Software hasn't gotten better, no new categories that couldn't exist before have emerged, and consumers feel no productivity explosion.

He acknowledges that AI itself is real technological progress, but the link of "applying it to software production" shows no output. There are two possibilities: either productivity is being swallowed by industry structure, or the claim itself doesn't hold up. He leans toward the latter.

Interview screenshot

The Only Practical Increment: Using AI to Find Vulnerabilities

Stripping away all grand narratives, Blow acknowledges one thing is genuinely useful—using AI to scan code at scale for bugs. This belongs to the same category as linters and fuzzers, "scaffolding tools around the programming process," and as a new tool it clearly has value.

But he also cautions that recent problems aren't just about finding vulnerabilities but also about extremely poor disclosure processes, causing attackers to outpace patch updates. The tool itself is neutral; how it's used is another matter.

Linter is a static code analysis tool that scans source code without running the program, detecting syntax errors, style violations, and potential defects; fuzzer (fuzz testing tool) feeds large amounts of random or semi-random data into a program, observing whether it triggers crashes or anomalous behavior, thereby discovering edge-case vulnerabilities. Both belong to "scaffolding tools around the programming process"—they don't directly participate in feature implementation but automatically extend developers' perception of code quality. Blow categorizes AI vulnerability-finding in the same category, meaning its value lies in being embedded in existing engineering workflows as a new type of detection tool, rather than replacing developers' judgment and design. This analogy also implies AI's limitations in this scenario: tools can find problems in known patterns but cannot understand system intent or make architectural trade-offs.

Is This Another Bubble?

Blow says he's lived through at least two software bubbles as an adult: the dot-com era and the startup accelerator boom between 2010 and 2017. He mentions attending a Y Combinator demo day in 2010 and leaving midway because he thought the company ideas were too stupid. The current AI boom fits the bubble pattern, just with a bigger upswing than ever before.

One signal he observes is a lack of ambition: almost all software-side AI startups are "using agents to do something" that could be accomplished simply by opening ChatGPT and having an agent do it. You're riding on someone else's foundation, and even if you grow big, the foundation owner will take it away.

Interview screenshot

His Diagnosis and Prescription

The direction Blow offers isn't radical. If he were to improve the programming industry, he wouldn't advocate regulation but rather hopes to establish some kind of "programmer monastic order"—even if one in a hundred, specifically responsible for maintaining the scriptures of "why it was designed this way originally," resisting the dissipation of knowledge.

He also repeatedly emphasizes an overlooked engineering discipline: regularly stop and pay down technical debt. He often does this in his own projects. Push features to the point where they work and approach the goal, then pause, clean up, and continue. This compression almost never exists at the industry level because no one has the authority to say "let's stop and rebuild the system."

As for the ultimate form of AI coding, he offers a counterintuitive vision: the system that truly succeeds may not be today's large language model approach but a return to AlphaZero-style systems trained on rules and their own experience, with LLMs as a conversational interface bridge. "If it could really do what everyone says, it would be the most valuable company in the world—but I think the probability is less than half."

One-line Summary

Blow believes AI coding lacks judgment, tenfold productivity is unproven, and the industry's experience gap is the deep root of quality problems.

Paragraph Summary

In the interview, Jonathan Blow systematically criticizes the software industry's chronic problems and the boundaries of AI coding. He points out that software engineering has never established a real professional accountability culture, methodological trends cycle but engineering quality sees no substantive improvement. Rapid industry expansion dilutes average experience and breaks knowledge transfer, compounded by ever-increasing system complexity, causing software quality to stagnate long-term. Regarding AI coding, he believes current large models are essentially efficient search algorithms, good at rapid output but lacking judgment, with redundant generated code accumulating as technical debt, and the so-called tenfold productivity lacking evidence. The only practical increment is using AI to scan for vulnerabilities at scale. He warns that the current AI startup wave shows bubble characteristics and advocates for a "programmer monastic order"-style knowledge guardianship mechanism and the engineering discipline of regularly paying down technical debt.

#Jonathan Blow#AI coding#software engineering#vibe coding#technical debt#software quality#AI bubble
Share:

Related articles