[KongchangAI]
· 4 min read· 2,147 words

AI Agent Engineering in Practice: From Watching Code to Auto-Merging PRs

AI Agent Engineering in Practice: From Watching Code to Auto-Merging PRs

A Cursor engineer spent five months building verification and constraint systems so Agents went from being watched while coding to auto-merging PRs.

The article recounts how Cursor engineer Lauren Tan built trust in AI Agents over five months, evolving from "every output needs human review" to "waking up to find about 20 PRs auto-merged." The core argument: it's not model capability but an engineering constraint system built around verification—control skills that let Agents actually run apps, capture traces, and reproduce issues; Feature Maps that help Agents locate UI features; skills that encode engineering taste in Markdown and are maintained via evals; and the Dune architecture that replaces soft constraints with CI hard failures (banning useEffect, banning comments, process isolation checks). The expansion path must go local verification → cloud Agents autonomously producing PRs → auto-merge, with no skipping ahead.

AI Agents writing code is no longer novel, but it's still rare for senior engineers to trust them with pull requests—let alone wake up to find 20 PRs already merged into main. In a technical talk, Cursor engineer Lauren Tan gave a complete retrospective on how she navigated that trust curve over five months—from "watching every Agent output" to "Agents auto-merging PRs." The key wasn't how powerful the model was, but a system of engineering constraints built around verification.

The Trust Curve: The First-Principles Problem of Agent Engineering

Lauren opened with a management analogy: if you don't trust your subordinates, your only management mode is micromanagement—watching everyone's screen, checking every line of output. It's the same with Agents. When you don't trust them, you become the sole verifier, and that directly caps parallelism: you can't spin up 100 Agents when you don't yet trust a single one's output.

She mapped her five-month journey onto a curve that mirrors contribution volume: the first month had low output due to unfamiliarity with the codebase, and as confidence in the Agent grew, delivery speed climbed rapidly. Recently, monthly merged PRs reached four digits, and in one month she was nearing 800 PRs after just 12 days. She acknowledged that this number invites questions about code quality, but stressed it was the result of good Agent configuration, not laissez-faire.

There are no shortcuts on this curve. Lauren explicitly warned: don't jump from "watching one Agent" straight to "spinning up hundreds of cloud Agents"—that just burns tokens at enormous cost. Trust must be built incrementally.

Verification Capability: Can the Agent Actually Run?

Lauren believes the most important skill when working with Agents is verification—the Agent must be able to actually run code, capture CPU traces or heap snapshots, open an iOS simulator, and run and test the app the way a real user would to confirm whether it works. This doesn't guarantee elegant code, but at least it produces correct code—the foundation of trust.

She gave the example of Cursor's internal Agent window (codenamed Glass). Early in her tenure, she needed to optimize performance under an extremely tight deadline, so she opened Chrome DevTools herself to read flame graphs—something the Agent had no concept of. Send it a trace screenshot and it would confidently point somewhere and say "that's the issue," only for the fix to miss the real problem entirely. At this stage, the human is the sole verifier and the bottleneck of the whole workflow.

So she built the first batch of control skills: teaching the Agent to launch the app programmatically via the Chrome DevTools Protocol or Apple's simulator tools, capture traces, and perform actions. The code itself wasn't complex—the Agent could generate it—but the key was the idea: turning "how to verify" into executable instructions for the Agent.

Agent-developer collaboration workflow

Chrome DevTools Protocol (CDP) is a remote debugging interface provided by Chrome that lets external programs communicate with the browser over WebSocket to programmatically control page navigation, execute JavaScript, capture network requests, obtain performance traces and heap snapshots, and more. It's also the underlying foundation for automation tools like Puppeteer and Playwright. For an Agent, CDP means it doesn't have to rely on screenshots or human descriptions to "see" the app—it can directly read runtime state, inject scripts to trigger interactions, and collect precise performance data. This turns verification from subjective observation into a reproducible automated process and lets the Agent independently complete the run-observe-diagnose loop without a human present.

Feature Map: Showing the Agent Where Features Live

Runtime capability alone isn't enough. Lauren found that although the Agent could launch the Agent window, it had no idea what was in the UI—when a user reported "the PR tab on the right is lagging," the Agent would rummage through the codebase, fail to find the corresponding feature, and the skill was essentially useless.

So she added a file called Feature Map to the control skills, teaching the Agent how to reach every feature: describing from the user's perspective which sub-features exist in the sidebar, which shortcuts they map to, and even recording the DOM attributes needed to select elements via CDP. This change made the skills truly usable—even when faced with low-quality feedback in Slack channels (many people just drop a screenshot and a few question marks), the Agent could use the Feature Map to understand context, locate, and reproduce the issue.

She productized this idea into a plugin called Pstack (a play on YC CEO Garry Tan's Gstack, with "P" for potato), which includes a create verification skill and a maintain verification skill—the former automatically explores the codebase to generate an initial Feature Map, and the latter maintains it continuously.

The naming lineage of the Pstack plugin and Gstack

Feature Map is essentially an "app navigation manual" for the Agent, describing each user-visible feature's entry point, interaction method, and location method in natural language and structured entries. It differs from traditional component documentation: component docs describe code structure, while Feature Map describes how to reach and operate a feature from the user's perspective, including shortcuts, menu paths, DOM selector attributes, and more. Its value lies in bridging the gap between "codebase knowledge" and "runtime behavior"—even if the Agent understands the source code, it may not know where the user clicks to trigger it. Feature Map makes this implicit product knowledge explicit, enabling the Agent to map vague user feedback (like a screenshot) to specific feature modules and reproducible action sequences.

Skills and Evals: Encoding Engineering Taste

Lauren observed a typical Agent failure mode: after a user reports a bug, the Agent confidently says "it must be here," but the tool call log reveals it never read the code that should be affected. Such moments are the easiest way to lose trust entirely.

Her response was to distill every failure into a skill. A skill is essentially Markdown, but it can encode a great deal of instruction and context. She cited a community saying—giving the Agent a high-quality token prefix is like pulling it "into another latent space," letting subsequent predictions unfold along smarter patterns.

Maintaining skills relies on evals, which you can think of as unit tests for the Agent. Pstack's potato mode has an eval playbook: a coordinator Agent sets the scoring criteria, then spawns multiple sub-Agents to execute tasks in their own directories (directory names deliberately don't reveal that they're being evaluated, because Agents notice and change behavior), and finally another model acts as judge for cross-validation to avoid scoring bias. Since Cursor supports multiple models, the same skill set can be evaluated across a model matrix. She also uses Cursor's loop instructions to run evals repeatedly until everything hits full marks.

But Lauren stressed that maintaining skills depends heavily on taste and observation: you need to be like a pair-programming partner sitting beside the Agent, reviewing every tool call and thinking block, and finding exactly where it drops the ball.

Eval (evaluation) borrows from the idea of machine learning evaluation in the Agent engineering context, but the object is not the model itself—it's the Agent's behavior on specific tasks. An eval typically includes a task description, initial environment, scoring criteria, and execution flow. The "coordinator Agent sets criteria, sub-Agents execute, judge model cross-validates" pattern described here is a multi-Agent evaluation architecture designed to reduce subjective bias from a single model's scoring. Directory names deliberately hide the evaluation intent because Agents perceive context and adjust behavior (similar to how humans behave differently when "being observed"), which would contaminate evaluation results. Cross-model matrix evaluation tests the robustness of the same skill set across models, preventing skills from overfitting to one model's behavior patterns.

From Local to Cloud: Benny and Auto-Merge

Only after the verification system matured could she expand to the cloud. Lauren introduced an Agent called Benny: it receives all bug reports, spins up its own "computer" in the cloud running Cursor, and uses the same control skills to reproduce issues. Once it successfully reproduced a bug and simultaneously confirmed it was already fixed on main—just needed a new release—information that previously took an hour of back-and-forth between a human and an Agent.

Her roadmap is clear: first build local verification capability and observe Agent behavior → once trusted, expand to cloud Agents that autonomously pick up signals (like bug reports) and produce PRs → only then talk about auto-merge. Today she wakes up to find around 20 PRs already merged by Agents, and by the time she reviews them they're already on main—and the quality is good.

Rewrites and Constraints: Hard Guardrails in the Dune Architecture

Lauren offered a rare fair defense of "rewriting the app." She believes brownfield apps (existing codebases) actually perform well with Agents if guardrails are in place; the real risk and opportunity lie in greenfield apps—like internal prototypes quickly vibe-coded where humans never read the code, the Agent picks the path of least resistance, and the codebase eventually spirals out of control.

Her solution was to rebuild the architecture for such apps (codenamed Dune, think "an Electron version of Next.js written for Agents"), turning constraints into CI-level hard failures:

  • Feature directory cohesion: all code for a feature lives in a single directory, so 80% of the Agent's work happens within that directory
  • No useEffect: one of React's biggest pitfalls, CI fails immediately
  • No code comments: she found that 99% of the Agent's comments recorded irrelevant historical information, and it would even write a criticism aimed at one PR as a permanent rule
  • Process isolation checks: dependency graphs ensure the renderer thread doesn't accidentally import main-thread code, avoiding dropped frames when the 16ms frame budget is blown by IO or heavy computation
  • Static analysis and lint: solidify observed bad patterns into checks

She specifically distinguished levels of enforcement: codebase architecture is the strongest constraint, followed by static analysis and compiler diagnostics, while rules, skills, and Bugbot (Cursor's CI code review tool) are "soft constraints"—the Agent may forget or apply them inconsistently. If you rely only on soft constraints and human code review to hold the codebase together, it will eventually spiral. She even noted that one reason Rust is regaining popularity is that its compiler is strict enough—as long as the Agent doesn't write unsafe blocks, passing compilation is basically trustworthy.

The signal is simple: when you keep leaving the same comment on PRs, that's an anti-pattern—turn it into a lint rule, a CI failure, or eliminate the problem at the root.

The design principle that the shortest path is the best path

Brownfield vs. greenfield are metaphors from software engineering describing a project's starting point: brownfield refers to existing systems with a large amount of legacy code, architecture, and historical decisions; greenfield refers to new systems built from scratch with no legacy baggage. The article's core judgment is that brownfield apps, with their existing architecture and constraints, make it easy for Agents to work within guardrails; greenfield apps (especially rapid prototypes) lack constraints, so Agents tend to choose the path of least effort, causing the codebase to quickly spiral. This distinction corrects the intuition that "a rewrite is always better"—the risk of a rewrite isn't the rewrite itself, but whether the new codebase has machine-executable constraints built in from the start.

ROI and Organizational Effects

Faced with the question "not everyone has unlimited tokens," Lauren didn't dodge: she works at an AI lab, so tokens aren't a concern. But she reframed the issue as an ROI calculation—refactoring the codebase and building constraints up front does burn money, but if the goal is for Agents to write most of the code and keep the team lean, that investment buys you "even the dumbest Agent can write good code." She estimated that refactoring the internal app cost over 600 PRs of work—years for one person.

The benefits aren't limited to engineers. Under the Dune architecture, PMs and designers can submit fixes directly, and Lauren just reviews and approves. She described this strict set of constraints as "annoying for humans to write, but the Agent absorbs all of that annoyance."

On PR size, she said there's no hard limit—ranging from dozens to thousands of lines—but she encourages the Agent to split them, because Git history is an important source of context, and atomic PRs are easier to roll back and pinpoint.

Editor's Commentary

The value of this talk lies in reframing the question from "can an Agent write code?" to "can you turn your engineering standards into machine-executable verification and constraints?" Auto-merging PRs sounds radical, but it's an outcome, not a starting point; before that came Feature Map, control skills, eval loops, and CI hard failures—layer by layer, building trust. For most teams, the most realistic starting point may not be Pstack, but answering: when will that comment you keep leaving in reviews become a lint rule?

#AI Agent engineering#Cursor#Agent auto-merge PR#verification skills#Feature Map#eval#Dune architecture#CI constraints#AI code review
Share:

Related articles