Multi-Agent Systems Explained on a Whiteboard: Architecture, Costs, and Three Lessons

Multi-agent systems are becoming a popular architecture for AI applications, but many teams rush to adopt them without understanding where they fit—resulting in ballooning token bills and worse results rather than better. An educational YouTube video explains the motivation, wiring, costs, and engineering lessons of multi-agent systems clearly in the span of a single whiteboard. This article is based on that video, cross-referenced with public materials from Anthropic and OpenAI.

Why do we need multiple agents?

Anthropic defines a multi-agent system as: multiple LLM agents using tools in a loop, collaborating to complete work. Two keywords stand out in this definition—"loop" and "collaborate." A single agent's ability to call tools in a loop is the foundation discussed in the previous installment; the core incremental value of multi-agent systems lies in splitting work across multiple independently running agents.

The video gives three reasons to use multiple agents.

First, research tasks are open-ended. It's hard to predict in advance which steps will be needed, so you can't hardcode a fixed path. Traditional workflows suit scenarios with a determined process, whereas open-ended research is better handled by letting agents plan autonomously.

Second, context compression. Sub-agents work in parallel, each running in its own context window, then compress the most important information and return it to the main agent. This keeps the main agent from being overwhelmed by massive amounts of raw information.

Third, separation of concerns. Each sub-agent has its own tools, prompts, and exploration paths. According to the video's author, Anthropic's internal evaluations show that such systems excel at breadth-first queries—tasks that explore multiple independent directions simultaneously.

One notable data point: in one test, a combination of Opus 4 as the main agent and Sonnet 4 as sub-agents outperformed Opus 4 alone by 90.2% on Anthropic's internal research evaluation. This suggests that a heterogeneous pairing of "strong main brain + fast sub-brains" may outperform a single strong model.

Why does it work? The answer is mostly tokens

The mechanism behind multi-agent performance gains can largely be reduced to "spending enough tokens."

In Anthropic's BrowseComp analysis—a test for finding hard-to-find information—token usage alone explained 80% of the performance variance. The implication is straightforward: part of the improvement from multi-agent systems comes not from a smarter architecture but from greater computational investment.

Understanding this makes the cost issue that follows clear.

BrowseComp is a benchmark released by OpenAI in 2025 specifically to evaluate a model's ability to "find hard-to-find, hard-to-locate information" on the open web. Questions are designed so that answers are short but require multi-step retrieval and cross-verification—for example, "Who is the third author of a certain obscure paper by a specific scholar?" Because the task itself depends heavily on search breadth and depth of information integration, it is often used to measure the retrieval and synthesis capabilities of agent systems. Anthropic cites this benchmark in its analysis precisely because it can reveal the proportional contribution of "architecture vs. computational investment"—the finding that token usage alone explains 80% of performance variance shows that, on open-ended retrieval tasks, much of the gain from multi-agent systems is "trading more compute for more thorough exploration" rather than an intelligence leap from the architecture itself.

How the system is wired

The video describes a typical wiring of a multi-agent research system through a complete flow:

A user's query first reaches the main agent, which plans the research approach and saves the plan to memory. There's an engineering detail here: the context window is truncated after 200,000 tokens, so the plan needs to be persisted in external memory.

The main agent then creates a number of sub-agents—no fixed limit—each assigned a specific research task. Sub-agents search the web, evaluate results, and return findings. The main agent synthesizes the returned results and judges whether more research is needed. If so, it can create more sub-agents or adjust its strategy.

When there is enough information, a citation agent is responsible for finding the source of each claim, ensuring all assertions can be traced to sources, and finally the result is returned to the user.

The cost ledger: 4x and 15x

The cost of multi-agent systems is very evident in token consumption. According to data cited in the video, agents use about 4x the tokens of ordinary conversations, while multi-agent systems use about 15x.

This means the task itself must be valuable enough to justify paying this price for performance gains. The video explicitly points out when it doesn't apply: when agents must share the same context, or are highly interdependent, multi-agent is a poor choice. Most coding tasks have lower parallelism than research tasks.

Scenarios suitable for multi-agent systems share several characteristics: high task value, high parallelism, information volume exceeding a single context window, and the need for multiple complex tools.

Context window refers to the upper limit on the total tokens a model can process in a single inference—it is both the model's "working memory" and a hard cost boundary. The original text mentions truncation after 200,000 tokens, meaning the main agent cannot keep all sub-agents' raw returns in context—this is precisely the origin of the "context compression" motivation: sub-agents each digest massive raw information within their own windows and return only distilled conclusions, allowing the main agent's window to focus on planning and synthesis. Understanding this also explains why multi-agent token costs are about 15x those of ordinary conversations: each sub-agent has its own independent context window, effectively opening multiple "working memories" in parallel, each paying for system prompts, tool definitions, and conversation history.

Three engineering lessons

Early agents made some typical mistakes, such as generating 50 sub-agents for a simple query. The video summarizes three lessons.

Lesson 1: Teach the main agent how to delegate. Each sub-agent needs a clear objective, output format, tools and sources, and clear task boundaries. Vague delegation descriptions cause sub-agents to go off track.

Lesson 2: Scale effort to query complexity. Control scale through rules in the prompt. A simple fact lookup needs only one agent and three to ten tool calls; complex research may use more than ten sub-agents.

Lesson 3: Parallelize. The main agent launches three to five sub-agents at once, each using three or more tools in parallel. According to the video's data, this reduced research time for complex queries by up to 90%.

Two handoff patterns: agents as tools vs. handoffs

How do multi-agent systems hand off work? In OpenAI's Python Agents SDK, there are two of the most common orchestration patterns.

Agents as tools. A manager agent retains control of the conversation and calls expert agents as tools. Suited to scenarios where a single agent is responsible for the final answer.

Handoffs. A triage agent routes the conversation to an expert, who takes over subsequent processing of the turn. Suited to scenarios where the expert should respond directly to the user. The two patterns can also be combined.

Claude Code likewise has a sub-agent mechanism; they work within a single session, each running in its own context window with custom system prompts, specific tool access permissions, and independent permissions.

Another common misconception needs clarification: A2A (an open communication standard between AI agents) is neither a sub-agent protocol nor a tool-calling protocol. The video suggests that sub-agents and tool calls should use framework-native primitives or MCP.

MCP (Model Context Protocol) is a standardized protocol open-sourced by Anthropic in late 2024 to let LLM applications connect to external tools and data sources in a unified way. It decouples "tool providers" from "model callers": a tool only needs to implement an MCP server once, and any MCP-compatible client (such as Claude Code or various IDE plugins) can call it directly without re-adapting for each framework. The original text suggests prioritizing framework-native primitives or MCP for sub-agents and tool calls precisely because MCP solves the problem at the layer of "how models access tools," while A2A solves the problem at the layer of "how agents discover and communicate with each other"—different layers that cannot substitute for each other. Confusing the two leads to using the wrong abstraction layer and adding unnecessary complexity.

One-sentence summary and practical advice

The main agent splits open-ended work among sub-agents with their own context windows, and when the task is worth the extra tokens, the investment pays off. Use it for valuable, parallelizable work; skip it when agents must share context or are interdependent.

The most important advice is: start simple. Find the simplest solution possible first, and add complexity only when truly necessary. Multi-agent is not a default option but a tool worth pulling out only in specific scenarios.

One-line Summary

Multi-agent systems stack tokens for performance at roughly 15x cost, suited only to high-value, parallelizable tasks.

Paragraph Summary

Based on an educational video and cross-referenced with public materials from Anthropic and OpenAI, this article systematically lays out the boundaries of multi-agent systems. It first gives three motivations for adoption: open-ended research cannot be hardcoded into a path, sub-agents work in parallel to achieve context compression, and separation of concerns. It then reveals the essence of performance gains—in the BrowseComp analysis, token usage alone explained 80% of performance variance, showing that improvements largely come from computational investment rather than the architecture itself. On cost, agents use about 4x the tokens of ordinary conversations and multi-agent systems about 15x, making them suitable only for high-value, highly parallel tasks where information exceeds a single window. The article also describes the typical wiring flow, three engineering lessons (teach delegation, scale by complexity, parallelize), and two handoff patterns—agents as tools and handoffs—and finally emphasizes starting with simple solutions.