[KongchangAI]
· 3 min read· 1,671 words

AI Wrote 90% of the Code—How Do You Prove It Runs? The Claude Code Team Publishes a 39-Item Incident Checklist

AI Wrote 90% of the Code—How Do You Prove It Runs? The Claude Code Team Publishes a 39-Item Incident Checklist

Anthropic's team proposes a verification loop, shifting AI code from review to evidence-driven validation.

When AI agents write 90% of a team's code, human focus shifts from line-by-line review to defining done criteria and reviewing verification evidence. Anthropic's Claude Code team proposes a verification loop using end-to-end, property-based, and differential testing, a 39-item reliability checklist, and a commit gate system to ensure AI-generated code actually runs. The core principle is "actually run it," and low verification costs enable running dozens or hundreds of agents in parallel.

When AI agents write 90% of a team's code—and even the reviews are done by AI—a fundamental question emerges: the code reads fine, but how do you prove it actually runs? Following their code review process, Anthropic's Claude Code team published a sequel that provides the answer: verification. Human work shifts from reading code line by line to examining evidence.

The Verification Loop: From Solo Combat to a Hundred-Battle War

Author Osmani spent 14 years at Google, where he managed Chrome Developer Tools, and now works on Anthropic's Claude Code team. He did the math: without a verification loop, you only dare run one or two Claudes at a time, because you have to personally resolve every change. With a verification loop, running dozens or hundreds in parallel becomes possible. Behind this is a cost inversion—writing tests, building benchmarks, and setting up real environments used to be expensive across the board; now handing them to agents makes the cost extremely low. Once tests are generated, each run is just running software, with no further model inference costs.

If you get stuck on the same check repeatedly, turn the lesson into a hook or rule to prevent the problem from recurring.

When agents execute in parallel, the verification loop is the key mechanism for ensuring each subtask produces reliable output. Without a verification loop, humans must personally inspect every agent's changes, capping parallelism at the limit of human attention; with automated verification (tests, static checks, end-to-end runs), agents can try in batches unattended, and humans only need to review the verification evidence. This model shifts humans from "executors" to "definers of verification standards," making it possible to run dozens or hundreds of agents simultaneously. Note, however, that verification itself must be fast and low-cost, or it becomes the new bottleneck; verification rules should also be centrally managed and must not be modifiable by the changes being verified, to prevent cheating.

Three High-Value Test Types: Giving AI Eyes and Hands

The article names three test types worth investing in, using aircraft analogies: full-aircraft flight, random storms, and dual-aircraft comparison.

End-to-End Tests: A Passport That's Hard to Fake

Simulate a real user's complete flow—add an item to the cart, check out, reach the confirmation page. Its defining trait is that it's hard to fake: either the whole feature works or it doesn't, with no middle ground. Unit tests can all pass while the feature is actually broken, because neither the agent writing the tests nor the agent being tested understands the full task. When an end-to-end test fails, it provides concrete evidence with screenshots, not just a timeout error. The article describes this as giving agents eyes and hands—through browser protocols, emulators, or direct API calls, letting them actually run the system.

Verification means actually running the change

Property-Based Testing: Set the Rules First, Then Let Random Inputs Try to Break Them

The approach is inverted: instead of giving examples, set rules first. For example, "an account balance can never be negative," or "any data compressed and then decompressed must be restored." Once the rules are set, tools generate thousands of random inputs specifically trying to disprove them. This method dates back to 1999, starting with the QuickCheck tool. The beauty is that random inputs have no bias—humans miss edge cases, and the agents writing code make the same mistakes, but random generators don't. This is especially useful when rewriting legacy code, where spec documents have long diverged from actual behavior.

An account balance can never be negative

QuickCheck is a property-based testing tool developed in Haskell by John Hughes and Koen Claessen in 1999. Its core idea is to use types to drive random test data generation and allow users to declare general laws the program should satisfy as properties. QuickCheck attempts to disprove these properties with large numbers of random inputs, automatically shrinking failing inputs to minimal counterexamples. This paradigm has since been ported to nearly all mainstream languages, such as Python's Hypothesis, Java's jqwik, and JavaScript's fast-check. The fundamental difference between property-based testing and example-based testing is that example-based tests verify outputs for specific inputs, while property-based tests verify invariants across all valid inputs, making them better at finding boundary conditions and unexpected combinations.

Differential Testing: Let the Old System Be the Answer Key

In jargon, this is called "dual comparison": run the old and new systems on exactly the same inputs, and any inconsistency is suspect. Inputs can be randomly generated or sampled from real production traffic. Set rules before comparing—for example, whitespace differences don't count—and keep an approved-change list registering legitimate behavior changes. When migrating code at scale, this is the most reliable safety net.

Differential testing, also known as dual comparison testing, is a method that finds defects by comparing behavioral differences between two or more systems implementing the same specification under the same inputs. Its theoretical basis is that "independent implementations should produce consistent outputs"; if they don't, at least one is wrong. Differential testing is commonly used in compiler testing (e.g., GCC vs. LLVM), database migration validation, and protocol implementation consistency checks. In practice, note that: the input space should cover both normal paths and edge cases, combining random generation with production traffic replay; comparison logic needs clearly defined equivalence rules (e.g., ignoring whitespace, tolerating floating-point error) and a maintained approved list of known differences, so noise doesn't mask real problems.

The Core Principle of Verification: Actually Run It

Above theory sits one principle: verification means actually running the change. Test web changes with a Chrome extension, hand mobile to emulation tools, use computer control to click on desktop, and run backend services locally. Package these steps into a skill in the repository called Verify, as close to the relevant code as possible. The article's exact words: "This smoke test tool must be rock solid."

The 39-Item Incident Checklist: Turning Lessons into Checks

The most substantial part of the article is the 39-item reliability checklist. Each item has three parts: a trigger readable from the code change, a check, and the historical incident behind it. For example, one item checks "do old records contain the newly added field"—after adding a field, old records won't have it. Claude reads through the change and only checks items whose triggers fire, usually three to ten. No one needs to memorize all the incidents; the incidents have become a checklist. This is exactly what aviation learned over a hundred years: behind every checklist is a crash.

Behind every checklist is a crash

The aviation checklist system originated from the 1935 crash of the Boeing Model 299 bomber (the B-17's predecessor) during a test flight. The aircraft was complex to control, and the test pilot forgot to unlock the elevator lock during takeoff, causing the crash. The investigation concluded the aircraft itself was fine; the problem was that humans cannot reliably remember all critical steps. Boeing then introduced a pre-takeoff checklist,固化 key operations into an item-by-item confirmation process, significantly improving safety. Checklists subsequently became standard practice in high-risk industries, with core design principles including: driven by real incidents, concise and executable, clear trigger conditions, and regularly reviewed and updated. Reliability checklists in software engineering borrow this idea, turning historical incidents into automatically triggered checks.

Institutional Design: Making Verification Impossible to Bypass

The team also designed four details to ensure the system sticks.

First, the commit gate has three levels: off, observe, and enforce. Observe mode only records what would have been blocked without affecting anyone; enforce mode is for real, but the same change is allowed through after being blocked twice, preventing Claude from getting stuck at its own gate.

Second, every commit gets a verification stamp noting the verification method, one value of which is literally "unverified."

Third, if a check fails three rounds in a row, stop and call a human.

Fourth, turn lessons into rules. A problem must appear in at least two commits before it qualifies as a rule. Before establishing a rule, three agents are dispatched specifically to refute it; only if they fail does it count.

Turning lessons into rules

For flaky tests, the iron rule is: no skipping, no adding waits, no retrying until it passes; if you truly can't find the root cause, delete the test and write down why. There's also an anti-cheating design: review rules are read only from the main branch, never from the change under review. The article's exact words: "If a change can modify the check that judges it, that check is broken."

The Human Role Shifts: From Reading Code to Reviewing Evidence

With 90% of code handed to AI, where should human attention go? The article's answer: from reading every line of change to defining what "done" looks like, then reviewing evidence. This mirrors aviation exactly—engineers don't personally inspect every rivet; they define pass/fail criteria and look at flight data.

In internal evaluations, having an independent grader check outputs against a rubric improved task success rates by up to ten percentage points, with the hardest problems seeing the biggest gains.

The original was published on Anthropic's developer blog. The previous article on code review scored 488 points on Hacker News; this one is the sequel and perhaps even more worth reading.

One-line Summary

Anthropic's team proposes a verification loop, shifting AI code from review to evidence-driven validation.

Paragraph Summary

When AI agents write 90% of a team's code, the focus of human work shifts from line-by-line review to defining done criteria and reviewing verification evidence. Anthropic's Claude Code team proposes a verification loop using three high-value test types—end-to-end, property-based, and differential testing—along with a 39-item reliability checklist and a commit gate system to ensure AI-generated code actually runs. The core principle is "actually run it," and the extremely low cost of verification makes running dozens or hundreds of agents in parallel possible. The human role shifts to defining pass/fail criteria and reviewing evidence, just as aviation engineers don't inspect every rivet but look at flight data.

#Claude Code#AI code verification#end-to-end testing#property-based testing#differential testing#verification loop
Share:

Related articles