Coding Agents in Practice: Migrating a Website in 3 Days, Building a Rust Library in 5 — The Methodology Behind It

The Week 3 guest for Stanford CS146S was Lee Robinson — formerly VP at Vercel, then at Cursor, now an engineer working on model behavior at SpaceX. Since the lecture slides and transcript weren't made public, this post takes a different approach: reading through his personal website end-to-end, diving into the open-source repos he linked, and distilling the most relevant hands-on lessons about coding agents. A quick disclaimer: all the numbers below are self-reported by the author. I looked at the image compression library's repo but didn't compile or run it.

Migrating the Cursor Website from CMS Back to Code — in 3 Days

After the Cursor website redesign, all content was pushed into a headless CMS. Lee Robinson's take was blunt: with coding agents, the cost of adding an extra layer of abstraction has never been higher. Before, you could ask Cursor to update a page in one sentence. Now you're clicking through menus in a backend dashboard.

He laid out five hidden complexities that CMSes introduce: users have to manage content in two separate places; the site needs static pre-rendering, so draft previews require a whole separate mechanism; each language needs its own content copy, which required a custom plugin; it's expensive — CDN costs since launch had run over $56,000; and most critically, the content lives on a remote server, meaning neither humans nor agents can directly search or edit it.

Take the navigation bar as an example: the old setup required a CMS API call to fetch a JSON blob, and the code had no idea what links existed. Once it was written directly into the component, updating a link was a single sentence to the agent.

The migration execution is worth studying. He used Opus 4.5 to plan the migration; the agent suggested writing scripts to export content directly from the API, and about ten runs got 80% done — the remaining 20% took the most time. For pages that didn't match, he had the agent take browser screenshots and compare them to the live version, iterating until they matched. He also used sub-agents to parallelize refactoring, and took the opportunity to remove Storybook, which was barely being used.

What was originally estimated to take one to two weeks — possibly requiring a contractor — took just three days. Total cost: $260, consuming nearly 300 million tokens (most of which hit cache), 344 agent requests, 67 commits, and a deletion-to-addition ratio of more than 7:1. His conclusion: spending a few tokens to eliminate complexity is money well spent.

5 Days, $287, a Rust Compression Library — Zero Lines Written by Hand

Some skeptics argued that the website migration only worked because he knew React and Next.js well. What about a domain he wasn't familiar with? So he deliberately chose two things he expected agents to struggle with: lower-level Rust and compression algorithms. The constraints were strict: zero runtime dependencies, all algorithms implemented from scratch, and no handwritten code throughout.

Over a five-day holiday stretch, he built a Rust image compression library: 520 agent sessions, 350 million tokens, $287 total. Test code accounted for more than half the project — 964 tests, 86% coverage. It compiles to WebAssembly at just 159KB and can compress images directly in the browser. He says performance is close to a 30-year-old C library — again, these are his own claims.

One key observation: most people today use agents for tasks that take 1 to 30 minutes, but models are already capable of running unattended for over an hour. The longest single run in this project went for three hours. His workflow was to first hash out the approach in planning mode, then hand it off to a cloud agent — much of the work was done while he was away from his computer, kicked off on his phone and merged hours later. The two models also had different roles: Codex was better for long-running tasks, while Opus was noticeably stronger for writing.

The Core of Letting Agents "Run Long": Verifiable Outputs

The secret to running that long without going off the rails was giving the agent verifiable outputs. From the start, he had the agent set up automated checks — formatting, linting, and tests. If a check failed, the agent read the error and kept going; only once everything passed would it submit a PR. He reviewed most of the code himself, and PRs were also sent to Bugbot for an additional review pass.

He specifically warned against over-optimizing the workflow upfront. He started with zero rules, and only after the model made a few recurring mistakes (like using the wrong check command) did he add a short rules file and a few reusable prompts. Eventually these were packaged into a nine-step release command — covering everything from writing changelogs to tagging and publishing — with each step required to execute automatically rather than just prompting the user. This is the same idea as the "skills" concept discussed in the course: encode repeated workflows into files.

---

The concept of "verifiable outputs" has a more formal name in agent engineering: observability constraints. The core logic is that when an agent runs autonomously, it can't rely on intuition the way a human can to judge whether a step went right. So the standard for "correct" must be externalized into machine-checkable signals. Common signal layers, from low to high, are: compilation (static correctness), lint/formatting (style consistency), unit tests passing (behavioral correctness), and end-to-end or visual comparison tests (integration correctness). Higher layers produce signals closer to real requirements but are more expensive to run. Lee Robinson's Rust project used a three-layer stack of formatting + linting + tests. The advantage of this combination is that every layer can be automatically judged as pass or fail — no need to ask a human — so the agent can close the loop of "execute → read error → fix → re-execute" entirely on its own. This is also why test code ended up comprising more than half the project: tests aren't just a quality assurance tool, they're the infrastructure that gives agents their "waypoints."

---

From Model Training to Continual Learning

Lee Robinson's current work is in model behavior. He compares models to colleagues: a good colleague communicates well, knows when to ask for help, and can work through poorly defined problems. The first step is documenting that ideal behavior — some call it a model spec, others call it a constitution.

He used basketball as an analogy: pre-training is like reading every basketball book ever written — lots of knowledge, but you still can't really play. Then you show the model tons of examples of good play, like watching footage of great players and learning by imitation. Then you put it in simulated games and reward it when it plays well — that's reinforcement learning. Scoring comes in two flavors: practice problems with answer keys you can check, and a final exam on unseen problems. Some are graded automatically (did the tests pass?), others are scored by a separate model using rubrics.

Being smart isn't enough, though. Researchers spend a lot of time chatting with models during training, logging their quirks (like defaulting to platitudes, or cutting things so short that responses become incomprehensible), penalizing those bad habits, and then running A/B experiments to see which version users prefer.

On continual learning, his point is worth highlighting: today's models only learn during training — no matter how much you chat with them, the conversations don't make the model itself smarter. If you want the agent to remember something, write it into a rules or skills file so the agent can load it into context. He says this "files as memory" approach works surprisingly well, but it's still far from the genuine learning a new employee would do.

This connects directly to the course's upcoming topics: skills, rules for agents, and the guest lecture by Boris Cherny, the author of Claude Code. Boris has talked about training coding agents — starting from an open-source model that understands code, doing another training pass on code data, and using automatically generated rewards for reinforcement learning (does it compile? do the tests pass? is the problem solved?). The training environment should mirror the real product, using the same tools for reading files, searching code, and running terminals. He said this is how Cursor trained Composer.

---

The training pipeline Lee Robinson described corresponds academically to RLHF (Reinforcement Learning from Human Feedback) and its derivatives. The basic steps are: pre-train on large-scale text to get a base model, then use supervised fine-tuning (SFT) to teach instruction-following, and finally use a reinforcement learning phase to further align with human preferences using reward signals. His "footage of great players" maps to the SFT phase; "playing simulated games with rewards for good play" maps to the RL phase. Rewards come from two sources: automatically determined rewards (e.g., did the tests pass, did the code compile) and model-scored rewards, where a separate "reward model" evaluates output quality — a reward model that was itself trained on human-labeled data. Notably, he emphasizes that today's models don't update their weights during inference, which means any feedback a user provides in conversation only affects that context window and is never persisted to the model's parameters. This is the technical root of the practical advice: "write down what you want the agent to remember."

---

Documentation, Community, and "Personal Software"

Lee Robinson's notes include a piece dedicated to documentation, listing ten standards for good docs. Most are timeless — fast, readable, useful, polished, localized, accessible. Two stood out as new: AI-native and designed for agents. He notes that most traffic hitting docs today comes from agents, so anything solvable by a command shouldn't require clicking a button; every page should have a one-click option to copy as Markdown, and you should be able to swap the file extension to read the raw source; there should be a table-of-contents file for LLMs listing every page; and going one step further, publish your docs as an MCP service — even release benchmarks telling users which model works best with your product. This is the same direction the course's MCP content points: deliver knowledge directly to where agents can reach it.

Having run developer relations, he distills the developer experience into three things: community is a long-term investment, and 90% of your interactions should be helping people — only 10% should involve asks; education is the best marketing, second only to word-of-mouth from happy customers; product engineers should work backward from the desired experience to figure out what technology is needed, and if a tool isn't pulling its weight, cut it — the same logic as removing the CMS.

There's also a piece called Personal Software: with AI, anyone can build software that's just for themselves. When he became a new father, he built a small app to track his baby's sleep and feeding. His analogy: making software will be like cooking — once you've cooked for yourself, you appreciate a great restaurant all the more.

---

MCP (Model Context Protocol) is an open protocol released by Anthropic in late 2024, designed to give large language models a standardized way to access external tools and data. Think of it as the USB-C of AI integrations: before MCP, every AI product had to build custom adapters for each external data source (databases, docs, APIs). MCP defines a unified server-client communication standard, so any tool that follows the protocol can be called by any model that supports MCP — no redundant integration work required. Publishing your docs as an MCP service means agents can actively "look things up" while writing code, rather than relying on potentially outdated knowledge baked in during training. The "table-of-contents file for LLMs" Lee Robinson mentioned is typically called llms.txt — an emerging informal industry convention where a structured plain-text index is placed at a website's root, listing all pages and brief descriptions for agents that crawl documentation. It functions similarly to robots.txt for search engine crawlers.

---

Four Takeaways Most Relevant to This Course

• Put content and configuration back into code — agents can only change what they can see;
• Give them verifiable outputs like tests, linters, and benchmarks — that's what lets them run safely for a long time;
• Writing code is no longer the scarce resource — judgment is: deciding what to build and what to cut;
• Models don't learn from conversations — if you want an agent to remember something, write it into a rules or skills file.

One more reminder: this post summarizes his personal website, not the lecture itself. All figures are self-reported. I looked at the image compression library repo but did not run it. His posts are mostly short — well worth reading in the original.