2,500 PRs Shipped in a Month: The AI Software Factory Methodology Behind pstack

Matt Pocock sat down with poteto — creator of pstack, grokbot, and a contributor to Cursor — for a deep-dive conversation centered on a jaw-dropping number: last month, poteto pushed roughly 2,500 PRs to production, a claim that racked up nearly 3 million impressions on X. But the real value of the conversation isn't the number itself — it's the reproducible methodology behind it: how to climb the "trust ladder" with AI agents step by step, and how to use verification, environment constraints, and determinism to remove yourself from the loop.

The Trust Ladder: From Human Middleman to Hands-Off Merging

poteto's story begins after leaving Meta. He took a month off and started a side project, naturally using AI to write code. But he quickly found himself spending hours micromanaging an agent — writing skill files, blindly guessing at outputs. That experience became the seed of pstack.

After joining Cursor to work on agent window performance optimization, the same bottleneck resurfaced: he had become a "human proxy," manually shuttling between the agent and Chrome DevTools — grabbing traces, taking snapshots, feeding results back. "I was the bottleneck" — that recurring realization became the starting point for his entire methodology.

The core logic of the trust ladder is this: the more you trust an agent, the more complex tasks you can hand it, or the more you can scale it out. The key to climbing higher isn't waiting for models to improve — it's transforming the environment the agent operates in. poteto emphasizes a point that's often overlooked: you can't change the internals of Claude Code, but you can absolutely change the codebase, give agents verification tools, and let them validate their own work.

Verification: The Single Most Important Skill in the Toolbox

If he could only keep one skill, poteto's answer is "verification." He uses a vivid analogy: verification gives an agent a pair of hands and a pair of eyes, letting it run code, interact with it, debug, grab traces, and take snapshots — just like a real user would.

Without verification, the "loop" doesn't hold. "The most important part of a loop being a loop is verification, because the agent can verify its own work — and that removes you from the equation." This was the first skill he built after joining Cursor, and it was the one that helped him take a critical step up the trust ladder.

Today, every app under Cursor and SpaceX AI ships with an auto-maintained Verification skill — it's become core infrastructure for the team. He even borrowed the Hill Climbing concept common in AI research — establishing a scoring rubric and letting agents iterate continuously toward improvement.

Hill Climbing is a classic heuristic from the optimization world: starting from the current state, move only toward neighboring states with a higher score, stopping when no better solution can be found. In ML evaluation, the concept has been extended into a paradigm of "continuously driving iteration with quantifiable metrics" — not seeking a one-shot optimal solution, but converging on the target through tight, repeated loops. poteto applies this to agent engineering by defining scorable evaluation criteria for each verification skill (e.g., render frame rate, interaction response time, assertion pass rate), having the agent auto-score after each change, and only keeping changes that improve the score. The key advantage is turning "is this good?" from a subjective judgment into a programmatically comparable number — so the agent can keep climbing in the right direction without human supervision.

The Gradient Between Deterministic and Non-Deterministic

poteto thinks of agents and skills as a gradient: some tasks are fully judgment-dependent (requiring stitching together multiple chunks of context to reason through), while others are highly mechanical (like refactoring code from one pattern to another).

He built a custom CLI specifically for verification skills, inspired by early anxiety about context windows. Rather than having every agent reinvent the wheel and write its own validation scripts, he extracted the deterministic parts into scripts or CLIs and embedded them in skills. This saves context, improves speed, and increases consistency.

"Skills are like a wrapper — a thin layer of lightweight instructions on the outside explaining how to use the built-in custom tools." The underlying move is compressing information and hiding complexity so agents do consistent things more reliably. The same logic applies to tech stack migrations: deterministic tools like Codemods and ASTs can handle mechanical transformations without involving an agent at all.

Codemod is a class of tools for batch automated source code refactoring, originally developed and open-sourced by Facebook for large-scale React API migrations. It works by parsing source code into an AST (Abstract Syntax Tree) — an intermediate representation of code structure as a tree of nodes — performing pattern matching and replacement on those nodes, and regenerating the code text. Compared to raw regex string substitution, AST-based transformations understand syntactic structure, handling nesting, scoping, and operator precedence correctly with almost no risk of breaking changes. In tech migration scenarios, a Codemod can mechanically rewrite thousands of files in seconds without any LLM reasoning involved — exactly what poteto means by "using deterministic tools for mechanical transformations."

Michelin Kitchen, Not Software Factory

poteto isn't fond of the "software factory" framing — it conjures images of assembly lines and ignores quality and craft. He prefers the analogy of a Michelin kitchen: moving from a home cook doing everything solo (chopping, prepping, cleaning up) to becoming a head chef leading a team. The point isn't division of labor for its own sake — it's making the whole greater than the sum of its parts.

The head chef is almost like a tech lead — not necessarily cooking every dish, but organizing the entire kitchen: thinking about ingredient sourcing, storage, and timing. Mapped to engineering, you're no longer writing code yourself, but your name and reputation are still tied to the final output. So how you set up your kitchen — building skills and environments — becomes the new raw material for building products.

This analogy also reflects his obsession with environment: a good codebase is one where making changes doesn't break things, which means lots of guardrails constraining both agents and humans to a narrow, correct path.

Encoding Errors Into the Environment

poteto believes the engineer's new job is to "spend time on the environment." He built an internal framework called Deon (similar to an internal Next.js inside an Electron app), with extremely strict lint rules and opinionated directory structure — there's effectively "only one way to do a given thing."

This thinking flows directly from his TypeScript background. His favorite TypeScript feature is type narrowing: starting from a broad type and shrinking the space through type guards and runtime checks. Constraining a codebase works the same way — limit the possible states, make it hard to write bad code.

The key practice is watching how agents fail. Every time he sees one go wrong, he steps back and asks: how can I turn this into a lint rule? How can I make the codebase structurally prevent this? Grokbot's first version consisted of 8 massive files with tens of thousands of lines each — it was precisely this kind of observation that drove him to refactor it into a feature directory discoverable by a registry. The environment then frees up human mental bandwidth while making agents feel like there's only one way to do things.

Inner Loop, Outer Loop, and the Chief of Staff Agent

The real secret behind 2,500 PRs is connecting the "inner loop" and the "outer loop." The inner loop is his agent engineers working on code toward an intent; the outer loop is the stream of bug reports and requests scattered across Slack, Linear, and X. Previously, poteto had to act as the middleman, manually routing external context to agents.

Connector tools like Grokbot (linking email, calendar, Slack, etc.) automate the outer loop. He borrows a management principle he learned at Netflix — "context, not control" — training agents to be self-sufficient rather than micromanaging every step.

In practice: multiple Grokbots continuously subscribe to channels, automatically creating tasks in Cursor's new Projects feature when bugs surface. Projects is essentially a cloud-based coordination agent (he calls it a "Chief of Staff" or "executive chef") — it doesn't do the work itself, but spawns, orchestrates, and manages sub-agents. When 30 related issues arrive simultaneously, the coordinating agent ingests the full payload, generates an optimal agent topology, and avoids redundant work across agents — sometimes multiple slightly different bug reports actually help reveal the bigger picture.

He also shared a daily technique: have one agent continuously monitor the codebase for bad patterns, but instead of fixing immediately, append findings to a doc as a "buffer." Coming back a few days later, you'll often find they're all the same underlying issue.

"Context, not control" is a core principle Reed Hastings distilled in Netflix's management culture book No Rules Rules: managers should fully convey strategic context, goals, and constraints to their teams rather than controlling execution through approvals and micromanagement — enabling high-density talent to make autonomous decisions with full information. poteto maps this principle directly to agent management: instead of manually writing detailed steps for each task, invest in skill files, codebase constraints, and verification tools — these form the infrastructure that carries context. Agents reading this context can make expected decisions without real-time human intervention. This also explains why he treats environment-building rather than per-prompt engineering as the core lever: once context is encoded into the system, every spawned sub-agent automatically inherits it.

Reviewing, Dark Factories, and the Limits of Verifiability

2,500 PRs obviously can't be reviewed one by one. poteto's approach is sampling — like a factory quality supervisor, doing a daily scan of PR quality and recurring bad patterns from agents. When multiple agents make the same mistake, the response isn't to fix that one agent — it's to step back and fix the environment (skills, constraints, lint rules, type system).

When environment constraints are tight enough, he's reached the point where agents can "merge their own PRs." Full Autopilot mode in pstack deploys a fleet of verification agents per PR to do fuzz testing: actually running the app, clicking around like real users, hunting for regressions and bugs, then self-correcting to a mergeable state. "Verification plus environment — those two together are what let me walk away."

That said, he remains clear-eyed about "dark factories." This isn't Vibe Coding-style forgetting about the code — quite the opposite: "code and environment both matter; garbage in, garbage out." The restaurant analogy is more accurate: even if you open a chain, the owner still drops by occasionally to taste the food.

On high-stakes domains like healthcare, law, and finance — and "one-way doors" (irreversible operations) — poteto admits there's no standard answer, but the core logic still comes back to verifiability: if work can be programmatically verified to high quality, a one-way door becomes, in a sense, a two-way door. He's optimistic about future languages that combine programming with proof for agent use cases (he mentioned Bend).

Fuzz Testing is an automated testing technique that feeds large volumes of random or semi-random data into a program to discover unexpected behaviors, crashes, and security vulnerabilities. It was first systematized by Professor Barton Miller at the University of Wisconsin in 1988. Traditional fuzz tools focus on input boundaries (malformed files, oversized strings), but poteto extends this idea in pstack to the UI interaction layer: deploying verification agents to randomly click around the interface, fill out forms, and trigger edge-case actions like real users — rather than only running predefined test cases. This approach surfaces regressions that are hard to catch with static analysis or unit tests alone, and is especially well-suited to frontend apps with complex UI layers and state management. Unlike traditional fuzzing, the "fuzz" here comes from an agent's generalized simulation of "real user behavior" rather than pure random byte streams — making it more targeted at real-world usability issues than traditional tools.

Skills as Process: Mining Gold from Conversation History

Discussing how pstack and Matt's skill framework intersect, both agree: skills are fundamentally about turning process into written words — "at the end of the day, it's just Markdown."

poteto's advice is to dig through your own conversation histories — they're a "treasure trove of context," the concrete embodiment of your process. Have an agent look through all the patterns you've used, the moments where you had to step in and correct things, and distill those high-level lessons into lint rules or reusable skills. The Recall skill he built in pstack does exactly that: compressing the "review past conversations" workflow into a single skill.

As models improve, skills will get smaller and more focused on the workflow itself rather than specific scripts and commands. "If there's any magic in it, it's in the words chosen and the phrases used" — which echoes poteto's fascination with language: natural language and programming language converge in the age of agents, because both are, at their core, about communication.