[KongchangAI]
· 4 min read· 2,333 words

Claude Writes 90% of the Code—So Who Reviews It? Inside the Seven-Layer Review System

Claude Writes 90% of the Code—So Who Reviews It? Inside the Seven-Layer Review System

Claude Code rebuilt review with seven automated layers, doubling PRs while cutting rollbacks 40%.

Anthropic's Claude Code team faced 90% AI-generated code and daily merged changes growing from 2,500 to 30,000. Instead of dropping review, they replaced a single gate with seven sieves: pre-PR agent self-checks and subagent review, multi-agent voting AI Reviewer, security review, CI, risk-tiered human approval, Merge Queue re-runs, and feature flags with fast rollback. They also built Cyan to handle online agent and CI loads. Capping review rounds raised one-round pass rates from 65% to 78%. Humans now focus on direction, high-risk code, reusable architecture, evidence, and accountability.

When AI-generated code accounts for ninety percent, line-by-line human review becomes nearly impossible. Anthropic's Claude Code team didn't scrap review—they rebuilt the entire process using seven layers of "sieves," doubling PR volume while cutting the rollback rate by over forty percent. How does this system work?

Background: When There's Too Much Code for Humans to Keep Up

Once agents started writing code, the volume of changes exploded. In Anthropic's monorepo, daily merged changes went from about 2,500 in May to over 30,000 by September. Claude Code has its own repo outside this monorepo, but the pressure is just as intense. At this pace, reviewing line by line as before is simply not feasible.

It wasn't just review that buckled—even Git couldn't keep up. The main branch history is nearly 100GB, and CI performs roughly 25 million checkouts per day. Pulling directly the Git way would generate over 600PB of traffic daily. The root cause is Git's design premise: it's distributed, every clone is a full repository, and it assumes clients work offline. But their clients are almost entirely online agents and CI jobs.

Git's design premise

So a team of seven spent five months building their own version control system, Cyan, plus a code review tool, Antique, and migrated the entire monorepo over. The infrastructure is 200 times smaller than the original Git mirror, and checkouts went from minutes to seconds.

Redefining the Goal of Review

The Claude Code team argues that "whether anyone read this code" is a very crude proxy metric. The real question is: what could go wrong with this change, and which check should catch it—automated or human?

The timing of review should also shift earlier—before the PR is opened, not as the final gate before merge.

They set a goal: most PRs should merge with at most one person involved, and that person can be the author themselves, while rollbacks and incidents must not increase. They track three numbers: human approvals per PR, human comments, and time PRs spend waiting. Rollback rate and incident rate are guardrails—if they rise, it means the time was just spent elsewhere.

Saving only on one end while the other end grows is no saving at all

The original article uses a metaphor: it's not a single door but a series of sieves. Each layer lets some issues through, but few issues can pass through all layers at once—and whatever slips through is caught by fast rollback.

Seven Sieves: From Before the PR to After Launch

Gate One: Before the PR, Let the Agent Self-Check

Before opening a PR, Claude runs its own checks—tests, builds, or a screenshot compared against the design mock—and produces the output rather than just saying "done." Then it spawns a subagent with fresh context to review the change. Many also run internal skills: one to find bugs, one to simplify the change, one for end-to-end behavior checks.

The creator of Claude Code has said that Claude being able to verify its own work is probably the single most important thing for getting good results—he estimates it makes the final output two to three times better.

If the same issue keeps getting flagged, fix it at the source: write it into a skill, add a lint rule, or update Cloud.md and review.md. Cloud.md describes the project's tech stack, conventions, goals, and what not to do; it's loaded on every review, so it must be short. Review.md is written by the code owner, telling reviewers what matters most in that area. It also limits nitpick-style comments and requires each comment to cite specific files and locations as evidence.

Subagent is an independent Agent instance launched outside the main conversation, with a fresh context window. It receives only the instructions and files needed for the current task and does not inherit the main conversation's history. Its value lies in "perspective isolation": the main Agent just finished writing the code, and its context is full of its own reasoning and design intent, making it prone to being overly lenient with its own output. A subagent without that baggage re-reads the diff like a stranger reviewer, more easily spotting missed edge cases, unhandled error branches, or inconsistencies with the rest of the repo. In the Claude Code workflow, subagents are used specifically for the "self-review" step, outputting a structured list of findings that the main Agent then decides to fix or explain.

Gate Two: AI Reviewer Takes the First Pass

Before a human looks at any PR, it goes through Claude's review, which examines the change in the context of the whole repo and the PR's intent: does it solve the problem, does it fit the system, did it break anything else, and for bug fixes, does it address the root cause. After launch, the share of PRs with substantive internal review comments rose from 16% to 54%. Large PRs over 1,000 lines mostly have findings; small PRs are mostly clean. Depth can be set to quick, standard, or deep.

Specifically, several agents hunt for bugs simultaneously, then another set of agents votes on each finding—only findings that get enough votes are surfaced, and purely stylistic or cleanup comments are discarded. These filtering rules were all A/B tested. Surviving comments are labeled by severity as important, nit, or pre-existing, and every comment must be addressed—either fixed or closed with a written rationale.

Often the agent does the fix overnight

Often the agent does the fix overnight, and the author returns the next day to find the PR has already been through another round.

Gates Three to Five: Security Checks, CI, and Human Approval

In an independent security review, a single unresolved bug or security comment blocks the PR from merging. Regular CI, lint, type checks, tests, and builds are all mandatory, and the PR template must explain how it was tested and how to revert it.

Human approval is tiered by risk. Low-risk changes (docs, copy, internal features, new features hidden behind a feature flag) require no human as long as automated review and security checks pass. Ordinary product changes and sensitive areas like caching, auto-update, and system prompts require one person. The highest-risk code and a few designated paths require two. The unit is the change, not the author; a classifier assigns the tier, and a second classifier decides whether an extra person is needed. Paths marked sensitive by owners and all CI workflow files always require a human.

There are also several hard rules: only authors on the trust list can use the lightweight path; agent approvals cannot substitute for a human; the person directing the agent counts as a reviewer; and approval is valid for one commit only—a new commit requires re-approval.

Feature flag is a mechanism that decouples enabling a new feature from deploying code: the code may already be merged into main and deployed to production, but the feature is off by default and only activates under specific conditions (internal users, specific tenants, a percentage of traffic). It serves two purposes in this review system. First, it lowers the review risk tier—a new feature behind a flag won't affect existing users even if it has bugs, so it can take the lightweight path with no human approval. Second, it provides the ability to "turn off" a problem without rolling back code; during an incident, changing one config stops the bleeding, with no need for an emergency commit revert. This is also the technical prerequisite for the "fast rollback" sieve.

Gates Six and Seven: Merge Queue and Fast Rollback

The Merge Queue re-runs checks on the latest main to prevent two individually passing PRs from breaking main together. New features hide behind feature flags and are used internally first; at public launch, latest ships before stable. The final gate is fast detection and fast rollback: a flag can be turned off without redeploying. Claude Code has a weekly rotating on-call, and incident postmortems ask which layer should have caught it. They also admit they're missing a circuit breaker—automatically tightening approvals when the rollback rate rises.

A colleague said slowing down does reduce errors, but mainly because less gets done—they'd rather find and fix problems faster.

Merge Queue solves the "each passes alone, fails together" problem. CI typically only validates a PR's build and tests against some main snapshot, but two PRs that are fine individually can turn main red after merging due to changes to the same function, the same config, or semantic conflicts. Merge Queue lines up pending PRs, takes the one at the front, merges it with the current latest main into a temporary branch, re-runs the full check suite on that combined result, and merges only if it passes—otherwise it goes back to the author. This keeps main always buildable and testable, at the cost of an extra CI run per PR.

Controlling Noise: The Cost of Agents Out-Reviewing Each Other

Catching more isn't enough—noise must be managed too. The team looks at how authors respond to each comment: fixed, said it was wrong, said it was intentional, or ignored. They also test against known-clean PRs, counting how many comments might induce people to make harmful changes. The priority is clear: catch real bugs first, then reduce review noise.

When the author is an agent, noise costs more. A human skims past an irrelevant nit in seconds, but an agent dutifully fixes it—adding a commit and another review round. Agent writes, agent reviews, and they can loop endlessly. One CLI tool PR went back and forth through review and revision over 50 times in a single night before the author called it off. Checking with git blame, about 59% of the code the reviewer repeatedly flagged was already there in the first round—it rarely found new bugs, just dug up a few old ones each round.

Running extra rounds was the team's own habit

Now they look at pass rounds and recall together and added four rules: after three rounds, ask the author to rethink the approach; afterward, only look at new changes; report a class of bug rather than a single line; don't repeat comments already made. The results are clear: PRs passing in one round rose from 65% to 78%, those needing five or more rounds fell from 8.9% to 1.5%, and average review count dropped from 2.08 to 1.32.

Recall here refers to the proportion of all actual problems that the reviewer catches, as opposed to precision—how many of the comments raised are real problems. Watching only rounds would tempt the team to loosen the reviewer to converge faster, but looseness misses bugs, so the team watches "pass rounds" and "recall" together: fewer round trips without sacrificing detection ability for speed. Testing false positives on known-clean PRs and using git blame to verify whether comments point to code newly introduced this round make both metrics measurable, avoiding optimizing one while quietly worsening the other.

What's Left for Humans: Five Things Only People Do Well

Pulling a senior engineer out of deep work just to catch a trivial error costs far more than an automated review. So humans should stay where only humans excel:

  1. Decide whether to do it—direction, taste, and scope.
  2. Review high-risk code—billing, auto-update, and CI workflows.
  3. Review architecture others will copy.
  4. Read the evidence—test output, commands and their returns, screenshots, reviewer findings. If it's unclear what the checks would miss, demand a real review.
  5. Own the outcome—every shipped change has a responsible person.

How the Team Learns the Code: Review's Second Role

With machines reading most of the code, how do people learn? Want to understand some area? Ask Claude to write an explainer for it in minutes, kept consistent with the current code; some even have Claude make a small page that visualizes that code. A one-line copy change teaches little, but an explainer for the module you're changing teaches a lot.

There's also a norm: if a change you merged causes a problem, you usually fix it. This works because changes are small, revertible at any time, more risky means more checks, regressions surface fast, and there's no blame—postmortems only look at the system.

Implementation Checklist and Direction

If you're following along: give the agent checks it can run itself before opening a PR; write a short Cloud.md plus a review.md; hook up an AI reviewer and security review to every PR; require all comments to be addressed before merging; cap review rounds; tier approvals by risk, with auth, billing, CI, and release processes always requiring a human; bot approvals can't substitute for a human; after merging, use feature flags, staged rollouts, and one-click rollback; and turn incidents into evals.

The public product currently only gives suggestions and doesn't approve on people's behalf, but in their own repo, low-risk changes can already merge on automated checks alone—and that line keeps moving. The goal has never changed: less human time per change, with no more rollbacks than before. Whatever slips through, add the information the model was missing to Cloud.md.

One-line Summary

Claude Code rebuilt review with seven automated sieves, doubling PRs while cutting rollbacks by forty percent.

Paragraph Summary

Facing ninety percent of code generated by AI and daily merged changes growing from 2,500 to 30,000, Anthropic's Claude Code team didn't scrap code review—they turned "one door" into "seven sieves": pre-PR agent self-checks and independent subagent review, multi-agent voting in the AI Reviewer, independent security review, regular CI, risk-tiered human approval, Merge Queue re-runs, and feature flags with fast rollback as the safety net. They also built their own version control system, Cyan, to handle online Agent and CI workloads. To counter noise and infinite loops from agents reviewing each other, the team capped rounds, deduplicated comments, and reported only new changes, raising the one-round pass rate from 65% to 78%. Human value is narrowed to five things: setting direction, reviewing high-risk code, reviewing reusable architecture, reading evidence, and owning outcomes.

#Claude Code#code review#AI-generated code#Anthropic#feature flag#Merge Queue#agent review
Share:

Related articles