The Cursor Swarm Experiment: What AI Rebuilding SQLite from Scratch in Rust Reveals About Cost

A Cognition-Shattering AI Collaboration Experiment

Cursor recently published a set of internal Swarm experiments whose results may change our ingrained assumptions about "how to make AI better at writing code." The experimental setup was extremely demanding: multiple batches of AI agents were tasked with implementing a database engine from scratch in Rust—without access to SQLite's source code, test suite, executables, or the internet—relying solely on an 835-page official documentation.

It's worth first clarifying where the concept of a "swarm" comes from. A swarm is a form of Multi-Agent System, inspired by the collective behavior of social organisms like ants and bees—each individual has extremely limited capabilities, but through local information exchange and coordination, complex abilities far exceeding those of any individual emerge at the group level. In AI engineering, a swarm uses role division (planners, executors, reviewers, etc.) to let dozens or even hundreds of agents process tasks in parallel, thereby breaking through the inherent limits of a single large model's context window and reasoning depth. This is a fundamentally different technical route from simply "feeding" more information to one giant model.

This is no ordinary CRUD application. SQLite is a complete relational database engine that must handle complex processes such as query parsing, query optimization, transaction management, and concurrency control. Even more demanding is that Rust is renowned for its strict compile-time checks—for code to compile successfully, it must satisfy ownership, borrowing, and lifetime rules, while the correctness of business logic must be separately verified through testing.

The reason SQLite was chosen as the touchstone is that it is the most widely deployed database engine in the world, embedded into nearly every smartphone, browser, and operating system. It adheres to ACID transaction properties and is renowned for its "single-file, zero-configuration, serverless" architecture. Although its source code is only about 150,000 lines of C, it contains a complete SQL compiler, a bytecode virtual machine (VDBE), a B-Tree storage engine, and a page cache manager. Making AI rebuild such a system without seeing the source code forces it to truly understand database theory, rather than relying on pattern matching to copy existing implementations—and this is precisely what gives the experiment its value.

In the end, both AI teams with different configurations passed the SQL Logic Test that Cursor had held back. But what truly deserves attention is not "whether AI can write a database," but rather the enormous cost difference caused by different model role assignments—and behind this lies the economic logic of AI engineering.

A 7.9x Cost Difference: The Economics of AI Model Role Division

The core of the comparison was two different model allocation approaches under the same swarm framework:

• All-GPT-5.5 configuration: Frontier models used for every stage, with a total cost as high as $10,565
• Hybrid configuration: Opus 4.8 for planning, Composer 2.5 for execution, costing only $1,339

The two differ in cost by about 7.9x, yet both passed the same tests. The key insight from this data lies in the mismatch between token consumption and cost.

In the hybrid configuration, the Workers (executors) consumed the majority of tokens, but their cost was only $411—about 30% of the model cost. Meanwhile, the planner used fewer tokens but cost $928. In contrast, in the all-GPT-5.5 configuration, the Worker stage alone cost $9,373.

This points to a counterintuitive conclusion: deploying expensive frontier models to every execution stage is not necessarily the most economically sensible choice. In large-scale software engineering, the stages that truly require frontier intelligence are actually very few—initial task decomposition, architecture design, and trade-off decisions. Once these uncertainties are eliminated and turned into clear instructions, the subsequent execution work can be entirely handed off to cheap, fast models.

This is a direct reflection of large models' pricing mechanisms. Currently, mainstream large models charge by token, and the unit price of frontier models (such as top-tier reasoning models) is often several to dozens of times that of lightweight models. At the same time, reasoning-intensive tasks (such as architecture planning) trigger longer chains of thought and higher output token consumption, which explains why the planner used fewer tokens yet spent more money—it called upon the most expensive "brain." Allocating intelligence on demand is essentially a form of compute cost optimization.

Three Disasters of AI Team Collaboration

Many teams are building agents in the hope of letting AI automatically turn requirements into code. But as soon as a task becomes complex, AI tends to hallucinate or get stuck in infinite loops. The common approach is to frantically expand the context window and let the model handle more tasks. But Cursor's experiment suggests: the direction may be backwards.

When hundreds of AI agents start working simultaneously like a team, Cursor found that AI teams' failure modes are completely different from those of humans, and encountered three major disasters.

Split-Brain and Planner Hijacking

The first problem is "split-brain": two planners unaware of each other will redundantly design the same concept in different places. Cursor primarily solved this through prompt constraints—key designs must be decided by the planner itself, and a delegated subtree cannot re-decide the same issue.

Even trickier is "planner hijacking": two planners, though fully aware of each other's existence, still keep modifying the same set of files back and forth. To address this, Cursor introduced a shared design document, references with compile-time checks, and a Reconciler responsible for coordinating conflicts.

Merge Conflicts

AI writes code fast but handles merge conflicts terribly: it either directly overwrites others' code or throws away all its own changes. Cursor's solution was to introduce a neutral third-party AI that writes no business code and whose sole job is to handle merge conflicts—essentially a fully automated "merge butler" within a human team.

Monster Files

There are always some core files into which everyone crams code, eventually turning into monstrous files of tens of thousands of lines. Human engineers instinctively avoid these, but AI is not afraid and will keep cramming until the system is dragged down. Cursor gave Workers permission to tag bloated files—once tagged, the system locks the file and forbids commits, then dispatches a dedicated AI to break it down into smaller modules.

From Git to Thousands of Commits per Second in Infrastructure

Supporting this collaboration is a whole new infrastructure. The previous Git-based browser swarm peaked at about 1,000 commits per hour, while Cursor's new system raised the peak to about 1,000 commits per second. To handle such an astonishing write density and implement dedicated coordination mechanisms, the team redeveloped its internal version control system (VCS).

Here it's important to understand why Git couldn't hold up. Git uses content-addressable storage and a snapshot-based commit model, designed for the low-frequency, asynchronous collaboration of human developers—even in the busiest open-source projects, commits happen on the order of minutes or even hours. When write frequency suddenly leaps to thousands per second, Git's file locking mechanism, the serialization of reference (ref) updates, and background garbage collection (GC) all quickly become fatal bottlenecks, and can even trigger data races and index corruption. This is precisely the fundamental reason Cursor had to abandon Git and build a VCS from scratch—the design assumptions of existing toolchains can no longer match the work rhythm of an AI swarm.

The value of the framework improvements is equally striking: under the Opus 4.8 plus Composer 2.5 configuration, the new framework achieved 100% pass rate with 4,645 lines of engine code; whereas the same model combination in the old framework required 19,013 lines of code to score only 97%. This clearly demonstrates that the improvement comes more from advances in the swarm framework than from the absolute capability of any single model.

It should be noted that this experiment simultaneously changed multiple mechanisms including VCS, prompts, review, and conflict handling, so the entire improvement cannot simply be attributed to cognitive division of labor.

Letting AI Write Its Own "Pitfall Guide"

There was another interesting detail in the experiment—Cursor calls it the "Field Guide." This is a document written autonomously by the agents and automatically injected into every agent at startup.

The reason is: model weights are frozen, and if the pitfalls encountered during runtime aren't recorded, the next agent will fall into the same trap. This amounts to letting AI write its own pitfall guide, and what's truly worth recording is precisely the unexpected situations.

This mechanism actually touches on a fundamental limitation of current large models—at inference time, the model is "stateless," its parameters frozen after training completes (i.e., weight freezing), unable to learn from experience during use. The industry typically compensates for this with external memory, such as vector databases, retrieval-augmented generation (RAG), or persistent context documents. The Field Guide is exactly an embodiment of this "external memory" approach: it distills the pitfalls the group encounters in practice into reusable, explicit knowledge, enabling the entire swarm system to achieve a kind of "collective experience accumulation" without any single model updating its parameters.

Regarding error review, Cursor found that no single review perspective could catch all problems: some look at conversation logs, some at output results, some at the codebase. But layering mutually low-correlated perspectives covers more errors—just as autonomous driving doesn't rely on a single radar but fuses data from LiDAR, cameras, and ultrasonic sensors. Since review costs are far lower than the work being reviewed, the return on compute invested in review is extremely high. (However, these mechanisms currently have no publicly available independent ablation experiments or quantified benefits.)

AI Isn't Writing Code, It's Doing "Compilation"

From an engineering perspective, the most profound insight is: Cursor's swarm is essentially a "probabilistic compiler."

Traditional compilers (like GCC, LLVM) translate deterministic high-level languages into machine instructions, in a strict and semantically invariant process. Cursor's agent swarm, on the other hand:

• Input: A requirements document written in natural language (the 835-page SQLite manual)
• Planner: Performs "front-end compilation," parsing natural language into a task tree and architecture design
• Worker: Performs "back-end compilation," translating the architecture design into concrete code
• Conflict resolution, code review, file splitting: Equivalent to the various optimization passes in a compiler

The qualifier "probabilistic" highlights its essential distinction from traditional compilers. Compilers like GCC and LLVM are deterministic: the same source code under the same configuration always produces exactly identical machine code, with strictly conserved semantics. A large-model-based swarm, by contrast, is probabilistic—the same requirements document may produce structurally different code across two runs, and may even err. This is both its weakness (non-reproducible, requiring extensive testing and review as a safety net) and its unique capability (able to handle fuzzy, incomplete, natural-language inputs—precisely the domain in which traditional compilers are utterly powerless).

This analogy also echoes economist Coase's theory of "the nature of the firm": firms exist because internal coordination costs are lower than market transaction costs. When hundreds of AI agents work together, the coordination costs among them (merge conflicts, split-brain, review) rise sharply, and mechanisms like shared design documents, decision references, and neutral conflict-resolution agents are essentially all about reducing the internal communication overhead of this multi-agent system.

Takeaways for Agent Developers

This serves as a wake-up call for everyone building agents: don't just fixate on a model's "IQ"—think more about how to reduce the communication costs between agents.

The way software engineering collaboration works is also evolving rapidly: from writing line by line, to writing block by block in the Copilot era, to writing file by file in the agent era. And the future Cursor depicts is one where the basic unit of work becomes the "requirements document."

Cursor expects that accurately describing intent will become an increasingly scarce ability. The focus of senior engineers may gradually shift from personally handling every syntactic detail to defining goals, boundary conditions, architectural constraints, and acceptance criteria. But architectural ability won't disappear, because a high-quality specification itself depends on this knowledge.

Finally, an objective note is needed: passing the SQL Logic Test only represents query result correctness; it does not evaluate performance, index efficiency, disk and memory consumption, transaction behavior, or concurrency. Therefore this result cannot be equated with a complete replication of all the functionality, compatibility, and reliability of production-grade SQLite.

To sum it up in one sentence: AI hasn't ended software engineering—it has simply rewritten the rules of human collaboration in code.

Key Takeaways