Making AI Agents Prove "Done": The Verification Loop Determines the Scaling Ceiling of Agents

When the rumor of an AI agent wiping out a production database in 9 seconds is still circulating among developers, a more fundamental question surfaces: why should we trust an agent when it says "I'm done"? The answer given by Nick Nisi, Engineering Manager at WorkOS, and Zach Proser, Engineer on the Applied AI team, at the AI Engineer workshop is this—turn "done" from the model's self-report into reproducible evidence.

Why "I think it ran successfully" isn't trustworthy

The most dangerous thing about agents isn't a lack of capability—it's overconfidence. As demonstrated in the talk with the "midnight snack" waitlist page: after submitting an email, the UI immediately shows success and everything looks normal, but refreshing the page makes all the data disappear—because only the frontend was verified, not persistence.

Nick uses a counterintuitive detail to illustrate the problem: "This repo isn't for you to read the code—we barely read the code anymore." When one person manages 10 to 30 terminal sessions simultaneously, diff attention is severely diluted, and relying on line-by-line review as a safety net is neither realistic nor worthwhile. Zach quotes a line from Lauren Tan as the anchor for the entire workshop: "If an agent can verify its own work, nothing else matters; if it can't, nothing else matters either."

Their core claim is: status comes from records, not model prose.

The Three Layers of the Verification Loop

Layer One: Write Verification as a Deterministic CLI

The most actionable advice from the talk: if your web app doesn't yet have a dedicated verification CLI, this may be the highest-leverage thing you can do for your codebase. The snack tool they built live is the template, offering commands like doctor (checks Node, Chromium, session, build status), join (executes joining the waitlist), storage (checks localStorage), reload, snapshot, screenshot, and proof.

Why is a CLI better than a long prompt? Because LLMs themselves are nondeterministic—the same prompt given to 30 agents might produce 30 different deviations; whereas a deterministic check based on bash commands can compress the variance to a minimum. Nick mentions that teams shipping thousands of PRs per month (like Lauren Tan's) almost universally make this their top priority.

Playwright is Microsoft's open-source browser automation and end-to-end testing framework, supporting the three major engines Chromium, Firefox, and WebKit. It drives pages directly through the browser protocol, can wait for elements to appear, simulate user interactions, assert DOM state, and has built-in screenshot, video recording, and network interception capabilities. Compared to the earlier Selenium, Playwright's auto-waiting mechanism and cross-browser consistency make it better suited to serve as an agent's "programmable eyes and hands"—the agent doesn't need to rely on vision models to guess page state, but instead uses deterministic selectors and assertions to obtain reliable results. This also explains why the talk implements the proof command directly as a Playwright script: it turns "the page looks right" into "the code verifiably is right."

Layer Two: Playwright-Level End-to-End Assertions

The proof command is essentially a Playwright script: confirm the waitlist is empty → fill in the email → click the button → assert the status element contains "wait listed" → check whether the inner text contains that email. Anyone who has written E2E tests is familiar with this—the difference is that this script is now handed back to the agent as a tool, letting it run, judge, and fix on its own.

Here's an important warning: a screenshot of success is not proof. A screenshot may show the correct state, but the agent may have taken a path that isn't even reachable through the UI. You must ensure the full path from start to finish runs through.

Layer Three: Use System 1 Models for Semantic Judgment

When the confirmation copy changes from "wait listed" to "your seat is confirmed," strict text assertions will falsely report failure—this is exactly the fragility of hardcoded assertions. Zach's solution is to introduce a System 1 model (such as Cloudflare's Cleft) for fast semantic judgment: not checking whether the literal text matches one-to-one, but whether the result matches the intent. This borrows from the fast-thinking/slow-thinking distinction in psychology, filling in a middle layer between deterministic assertions and human intuition.

System 1 / System 2 originates from the dual-system theory proposed by Daniel Kahneman in Thinking, Fast and Slow: System 1 is fast, automatic, intuitive judgment, while System 2 is slow, deliberate reasoning that requires working memory. In the context of AI engineering, this distinction is often borrowed to describe model calls of different scales and latencies—small models for fast classification and semantic matching, large models for complex planning and reasoning. The talk introduces a System 1 model precisely for its low latency, low cost, and suitability for high-frequency calls: it doesn't need to compare copy word by word, only to judge whether "the current interface state matches the expected intent." This provides a flexible supplementary layer for hardcoded text assertions, preserving the speed of deterministic checks while avoiding false positives caused by minor copy changes.

From "Supervisor" to "Gardener": The Shift in Working Style

Zach describes his daily state as "gardening": cultivating multiple self-evolving codebases simultaneously, rather than painstakingly sculpting a single tree. He built two experimental tools:

• CTX (context): Originally a memory layer across agent harnesses, letting ChatGPT, Claude Code, Pi, and others share read/write memory; later it evolved into a centralized verification authority—after an agent finishes work, it connects back to CTX via MCP saying "I'm done, please verify," and CTX calls a preset verifier (such as querying the GitHub API to confirm the PR actually exists and merge checks pass) before marking the node complete; otherwise it returns the failed items to the agent.
• Riker: A "nanny-type" engineering agent that, after taking over a PR, monitors tests and review comments, verifies that fixes still align with the overall goal, and only calls for human intervention when needed.

The common idea behind both tools: reclaim the authority to define "being in a done state" from the agent's mouth into an auditable third party.

Failure Information Is the Agent's Fuel

An easily overlooked but extremely critical detail from the talk: writing good error messages is more important than ever. When the application throws an error, the one best equipped to interpret it and act on it is precisely the agent. If the failure message clearly explains why it failed and how it entered that state, the agent can self-heal. Conversely, vague error messages only lead it to guess blindly or fabricate.

This is also the origin of the "receipts" concept: every run, every screenshot, every video, every console log is data that can be reused for future evals. Zach's judgment is that, with the growing adoption of RSI (recursive self-improvement) harnesses, teams with clean historical run datasets will be able to do hill-climbing optimization faster than competitors.

Constraints Are Not Shackles: Give a Map, Not a Route

A misconception worth guarding against is turning verification into over-specification. The speakers emphasize that frontier models are already very good at finding their own path when given an endpoint, and may even discover edge cases you hadn't thought of; but if you hardcode every step, they become afraid to deviate and performance drops. The right approach is: give a map and arrival markers ("when this command returns this result, you've arrived"), not a prescribed path.

The live session also demonstrated a small "saboteur and detective" game: randomly inject a bug and let participants locate and prove it using only the snack tool. The game's design intent is to train the TDD-style loop of "reproduce first, then fix, then prove the fix works," and this red-green-refactor cycle applies equally to agents themselves.

What This Means for Teams

The implementation path the speakers give isn't complicated: put an agents.md (not claude.md) in the repo specifying which verification commands must be run after changes; wire the verifier into CI as the final gate; use hooks for local safeguards. At the same time, they acknowledge that there's currently no general-purpose tool that covers all applications—verification logic must be written for your own business, and this is precisely the differentiated asset teams can accumulate.

A more radical corollary: when the verification layer is complete enough, the importance of the source code itself declines. The talk mentions the case of the Bun team converting their project to Rust and then back—if there's a complete verifier, rewriting the code and proving behavioral equivalence becomes feasible.

During the transition period as agents move from "can write code" to "can self-evolve," verification is not the opposite of speed—it's the prerequisite for speed. As the speakers repeatedly emphasize: this isn't a 0-to-1 switch, but a gradual trust curve—every piece of reproducible evidence is another step up.

Background Supplement

MCP (Model Context Protocol) is a protocol proposed and open-sourced by Anthropic in late 2024, designed to let models or agents connect to external tools and data sources in a standardized way. It abstracts "tools" as discoverable servers, and agents initiate calls through a client, thereby extending capability boundaries without modifying the model itself. In the talk, CTX lets agents "connect back" and request verification—this is precisely using MCP to make the verification authority an independent service: the agent no longer declares completion itself, but calls an external verifier to obtain a trusted state. This design extends the protocol from "giving agents more tools" to "giving agents an auditable judge," which is key background for understanding the CTX architecture.

One-line Summary

Agent done states should come from reproducible evidence, not model self-reports; the three-layer verification loop is the prerequisite for trust.

Paragraph Summary

The article centers on the core question "why should we trust an agent when it says it's done," proposing that states should come from records, not model prose. The author introduces a three-layer verification loop: a deterministic verification CLI, Playwright-level end-to-end assertions, and semantic judgment by System 1 models, covering scenarios from hard checks to flexible intent matching. The article emphasizes that a verification CLI is the highest-leverage infrastructure investment for web apps, significantly reducing variance across parallel agents; the quality of failure information directly determines an agent's self-healing ability, and every run record is fuel for future evals. In terms of working style, the author advocates shifting from "supervisor" to "gardener," and through tools like CTX and Riker, reclaiming the authority to define completion into an auditable third party. Finally, it points out that verification should give an endpoint rather than a path—over-specification suppresses the exploratory capability of frontier models.