Jeremy Howard: AI Coding Is Like a Slot Machine — Your Wins Are Just Losses in Disguise

AI-assisted coding has been elevated to near-mythical status. GitHub Copilot, Cursor, and a growing ecosystem of agent-based tools have led many to believe developers are living in a golden age of productivity. But Jeremy Howard, renowned AI researcher and co-founder of fast.ai, has offered a sharp counterpoint: AI coding is more like a slot machine than a genuine productivity engine.

The Illusion of Control: You Think You're Coding, But You're Just Pulling a Lever

Howard's central argument is that AI coding tools give developers an "illusion of control." You can carefully craft your prompts, configure your MCP lists, fine-tune your skills and context — it all feels like you're in command.

But at its core, what you're doing isn't fundamentally different from pulling a slot machine lever. "You put in a prompt, and something comes back to you — like 'cherry, cherry' on the reels." When the result isn't what you wanted, your instinct is: "Let me tweak the prompt, add some more context, and pull the lever again."

This behavioral pattern exposes the underlying problem. You're not solving problems through deterministic logical reasoning — you're iterating inside a stochastic system, hoping the next roll lands in your favor.

Randomness + Intermittent Rewards: The Casino's Classic Formula

Howard points out that this process has all the hallmarks of a "loss disguised as a win": the outcomes are stochastic, yet you feel a sense of control, and occasionally you get that rush of "winning" — "Oh, I got it, I shipped a feature!"

This is exactly the mechanism casinos engineer into their machines. Intermittent reinforcement is the most addictive reinforcement pattern in behavioral psychology — the uncertainty of when you'll win is precisely what keeps you pulling the lever.

When AI occasionally produces an elegant solution, the sense of accomplishment masks all the time and effort wasted on the failed attempts before it. You remember the win; you forget the cost of repeated trial and error.

Background: Intermittent reinforcement was systematically studied by behavioral psychologist B.F. Skinner in the 1950s through pigeon experiments. He found that when rewards appear at unpredictable intervals ("variable ratio reinforcement"), subjects develop the most stubborn, hardest-to-extinguish behaviors — continuing to repeat the action even after long stretches without reward. This is the psychological foundation of slot machine design: players don't know which pull will pay out, and that uncertainty itself becomes the motivation to keep pulling. By contrast, if rewards are fixed and predictable (e.g., every 10th pull wins), the behavior is actually easier to stop. AI coding tools align closely with this pattern at the experiential level: model outputs are inherently stochastic, the same prompt can yield wildly different results at different times, and developers can never predict whether the next attempt will produce satisfactory code. This makes the impulse to "try one more time" psychologically isomorphic to gambling behavior.

The Data Speaks: Actual Software Delivery Has Barely Budged

Howard's most powerful argument isn't the analogy — it's the data. He referenced research his team recently conducted, concluding: "Nobody is actually creating 50 times more high-quality software than before."

The reality, he says, is that what people are actually shipping has shown only a "tiny uptick."

This finding directly punctures the industry narrative of "10x engineers" and "exploding productivity." If AI truly boosted coding efficiency by tens of times, software delivery rates should have jumped by an order of magnitude — but the real-world data doesn't support that claim.

There's an important distinction worth sitting with: the speed of writing code and the speed of delivering working software are not the same thing. AI can generate large volumes of code quickly, but that generated code still needs to be reviewed, debugged, tested, and fixed — steps that may neatly cancel out whatever time was saved during generation.

Research context: Howard's observations echo findings from several recent independent studies. A 2024 study by METR (Model Evaluation & Threat Research) found that in real-world open-source project tasks, AI assistance did not significantly speed up experienced developers as expected — and in some tasks, developers using AI actually took longer, because the cognitive overhead of verifying AI outputs was systematically underestimated. The gap between "code generation speed" and "delivery speed" is especially pronounced in large projects: acceleration during the generation phase tends to be consumed downstream by code review, integration testing, and dealing with "hallucination bugs" (code that looks plausible but contains subtle logical errors). This means evaluating AI productivity gains based solely on lines of code or subjective self-reporting has a systematic tendency toward overestimation.

This Doesn't Mean AI Is Useless — It Means It's Hard to Evaluate

Howard was careful to note that his critique "doesn't mean AI isn't useful." His position is genuinely measured: "God, it's really hard to tell."

That honesty is arguably the most valuable part of his message. He doesn't swing to the opposite extreme of "AI coding is a scam" — instead, he identifies the core issue: evaluation is hard. We lack reliable methods to measure how much real value AI actually delivers.

The combination of stochasticity and the illusion of control makes individual subjective experience deeply unreliable. You may feel like your productivity has skyrocketed — but that feeling itself might be a product of the slot machine mechanism. Getting to the truth requires objective delivery-based research like Howard's team conducted, not developer self-reporting.

What This Means for Developers and Teams

Howard's analogy offers a sobering counterpoint to the feverish enthusiasm around AI coding. A few takeaways worth considering:

• Watch out for addictive usage patterns. If you catch yourself repeatedly tweaking prompts and "pulling the lever one more time," step back and ask whether this is genuinely the most efficient path to solving the problem — or whether you've fallen into a loop of random reinforcement.
• Distinguish between feelings and facts. Individual productivity perception is easily inflated by occasional wins. At the team level, use objective metrics like delivery rates and defect rates to measure the real impact of AI tools.
• AI still has value — but calibrate your expectations. A small improvement is still an improvement. The key is not to be held hostage by marketing claims of "50x gains" — use these tools for what they're actually good at.

What gives Howard's perspective its weight is that it comes from a practitioner deeply invested in AI education and research — not a skeptic on the sidelines. When someone from inside the industry is warning you not to mistake losses for wins, perhaps it's time for the whole industry to take a closer look at the shiny slot machine in its hands.