700 Companies Tested AI Coding Agents: Code Volume Surged 30%, So Why Did Output Stay Flat?

More Code, But No More Software

Over the past two years, AI coding agents have been repeatedly packaged as a "superpower" for the software industry—give it an instruction, and it writes code and submits merge requests on its own, with almost no human intervention. By that logic, companies should soon be shipping more software. But a study covering more than 700 software companies delivers the opposite answer: code volume did go up, but the work actually completed barely changed.

The study was conducted by Fiona Shen and James Stratton of Harvard University, using data from engineering efficiency analytics firm Jellyfish, covering 2021 through March 2026, more than 2,000 engineers, and roughly 300 million work events—a sample size rarely seen in comparable research. The researchers used a "within-firm before-and-after" design: what the same company produced before adopting AI agents versus after.

The results reveal a striking disconnect. After adoption, lines of code rose 30% on average, commits increased 20%, and pull requests increased 23%—on the surface, a boom. But when the researchers turned to the tasks, stories, and epics recorded in Jira—the work items that "someone asked for and someone confirmed was done"—task completion rates showed no statistically significant change.

They also specifically investigated one possible explanation: were companies simply breaking tasks into smaller, simpler pieces? The answer was no—neither the size nor the complexity of recorded tasks changed noticeably. Code multiplied, yet software delivery stood still.

AI coding agents refer to AI systems that can receive natural-language instructions and autonomously complete tasks such as writing code, modifying files, and submitting merge requests. Representative products include programming assistants built on large language models (such as GitHub Copilot's agent mode, Cursor's Agent feature, and Devin). Unlike traditional code completion tools, agents have multi-step planning and tool-calling capabilities, enabling them to complete end-to-end development tasks with minimal human intervention. These tools entered enterprise development workflows rapidly after 2023, carrying expectations of dramatically improving software delivery efficiency—making them a key subject for evaluating AI's real impact on productivity.

The Bottleneck Isn't at the Keyboard—It's at the Review Desk

Where did all the extra code go? The paper pinpoints the mechanism in one sentence: "Efficiency gains were absorbed by constraints downstream in the production process." In other words, writing code was never the only hurdle—the problem arises after the code is written.

The most obvious chokepoint is code review. The study observed that the review stage actually lengthened after agents were introduced: pull requests required more rounds of revision, reviewers left more comments, and a research summary noted that review time per PR rose by about 49%. That number is easy to understand—if code volume rises 30%, someone has to read it, understand it, and judge whether that 30% is actually correct. And the machine writing the code doesn't get tired, doesn't get frustrated, and doesn't consider whether reviewers have time.

To use an analogy: no matter how fast you pour water into a bottle, the amount that flows out depends on the narrowest bottleneck. AI expanded upstream capacity, but downstream human judgment didn't scale in tandem. Understanding requirements, deciding what to build, reviewing whether it's reliable, integrating with existing systems—these steps still depend on human judgment, and they are the real limiting factors.

This also explains why both popular narratives—"AI will make programmers unemployed" and "AI will cause an explosion in software supply"—fail to hold up. The study doesn't say AI is useless; it says that once the cost of writing code falls, the center of value shifts from "writing faster" to "reviewing better," from "producing more" to "deciding what not to produce."

Code review is the quality-gate step in software development: after a developer submits a pull request (PR), other engineers read the code line by line, suggest changes, and confirm logical correctness and system compatibility before it can be merged into the main branch. It typically involves multiple rounds of back-and-forth revision and is a core mechanism for ensuring code maintainability and security. Review efficiency depends heavily on reviewers' time and cognitive bandwidth—reviewers must understand code intent and assess potential risks, a process that is difficult to fully automate. When upstream code output surges, review queues back up and review times lengthen, often becoming a structural bottleneck that limits overall delivery speed.

Using AI to Review AI-Written Code—Does That Solve the Problem?

A natural response is: if review is the bottleneck, then use AI to review. A substantial share of code reviews are indeed now performed with AI involvement, but according to the research cited, this is still considered a minority of cases.

This raises an uncomfortable question: if a piece of code is written by one AI and reviewed by another, where exactly does human judgment enter the process? If two models share similar blind spots, review may become a formal confirmation rather than genuine gatekeeping. The study doesn't offer a definitive answer, but it puts the question on the table—and in organizations chasing throughput, this question is easy to overlook.

A detail worth noting: this finding aligns with what many frontline developers have experienced—AI-generated code looks polished, but rework, explanations, and debates before merging have actually increased. The gains from productivity tools are often quietly eaten away by friction in organizational processes.

Before Drawing Conclusions, Let's Clarify the Limitations

Before treating this study as definitive, several boundaries need to be made clear.

First, this is a working paper that has not yet undergone formal peer review. Second, the data runs through March 2026, and this kind of technology changes every few months, so the conclusions have a limited shelf life. Third, the study presents firm-level averages—it's entirely possible that some teams use agents very well while others see almost no impact. Fourth, Jira is a reasonable measure of "work completed," but it's not perfect—some work isn't recorded in Jira, and some recording practices are themselves biased. It should also be noted that "no statistically significant change" does not equal "zero change."

There's also a detail involving a conflict of interest: Jellyfish itself sells engineering efficiency analytics tools. This doesn't mean the data is flawed, but readers should keep it in mind.

Working paper is a form of academic research circulated publicly before formal peer review, typically published on preprint platforms such as SSRN or arXiv, or in institutional working paper series. Its defining feature is that the research methods and preliminary conclusions are already formed but have not undergone independent anonymous expert review, so methodological flaws, data issues, or biased conclusions may exist. Working papers are often cited to rapidly disseminate new findings, but readers should treat them as "preliminary results pending verification" rather than definitive conclusions. In fast-evolving fields like AI, conclusions from working papers especially need to be assessed alongside subsequent peer-reviewed versions and replication studies.

The Real Question Has Changed

Back to the original premise: give companies a superpower that writes 30% more code, and software delivery doesn't budge. The surprise isn't that "AI is useless"—it's that the bottleneck was never at the code-writing stage.

The industry spent years believing that software's problem was that writing code was too slow. Now that code is nearly free, people are beginning to see clearly: what's truly scarce is the ability to understand, trust, and approve. The question is no longer "how much code can a machine write" but "how much code can a person read, dare to trust, and be willing to approve." At least for now, that answer hasn't changed.

The practical implications for companies are quite concrete: when adopting AI coding agents, focusing only on code output metrics can easily create a backlog of work waiting for review. What should really be scaled up in tandem is review capacity, requirement clarification, and integration processes—the human judgment steps that AI can't yet replace.