Multi-Agent Systems in 3 Minutes: From Stateless Models to Collaborative Architecture

Large language models are, at their core, a function — text goes in, text comes out. This seemingly simple description reveals the central constraint behind every agent system design. Understanding it is the starting point for understanding why multi-agent systems are built the way they are.

LLMs Are Stateless Functions

To understand agents, you first need to accept a counterintuitive fact: the model itself has no memory. When you call a model a second time, it has absolutely no idea what happened during the first call. Between two calls, the model saves nothing — it is stateless.

This characteristic drives all the architectural decisions that follow. Since the model remembers nothing, all "memory" and "context" must be re-supplied by your code on every single call. The model's capability boundary is clear and fixed. It will never proactively carry forward any state — all continuity is an illusion constructed by external engineering.

Models have a second limitation: their knowledge is frozen at their training cutoff date. Anything that happened after training was completed is unknown to the model. This is precisely why we need additional mechanisms to fill in its knowledge gaps.

Agents: A Loop Around the Model

If a model is just a stateless function, what exactly is an agent? The answer is straightforward — an agent is a loop in your code.

The loop works like this: your code first constructs a prompt containing the goal, memory, and a list of available tools; the prompt is then passed to the model, which makes a decision; if the model decides to call a tool, your code runs that tool and feeds the result back into the prompt; the loop continues until the model produces a final answer.

There's a key point that's easy to misunderstand: the model never executes code. It simply writes out a request like "I want to call this tool," and it's always the external code that actually does the execution. The model is responsible for decisions; the code is responsible for actions. The boundary between the two is crystal clear.

Memory and RAG: Both Are Data Retrieval at Their Core

The model's two limitations — forgetting conversations and having a knowledge cutoff — correspond to two solutions: a memory system and a RAG system. But look closely and you'll find they're both doing the same thing: your code finds the right data, then puts that data into the prompt. Memory and RAG are both, at their core, data retrieval problems.

Layered Memory System Design

Memory systems mirror the structure of human memory, organized into several layers:

• Working memory: Stores the current conversation, held in RAM, expires after roughly one hour;
• Episodic memory: Stores timestamped events;
• Semantic memory: Stores facts and the relationships between them, typically in a graph structure;
• Sensory memory: Stores multimodal content such as images and audio.

To find a relevant memory, the system computes a composite score. Similarity carries the most weight, recency contributes a smaller portion, and the result is multiplied by an importance factor ranging from roughly 0.8 to 1.2. This means importance can shift a memory's ranking, but it cannot determine whether a memory is retained in RAM in the first place.

RAG Retrieval Techniques

A useful approach in RAG is to first have the model draft a "hypothetical answer," then search around that answer. The more angles you search from, the more relevant text chunks you retrieve, and the higher the recall quality. This "hypothetical answer-driven retrieval" approach tends to outperform searching directly with the raw question.

RAG (Retrieval-Augmented Generation) is the mainstream solution for addressing the model's knowledge cutoff problem. The core workflow is: split external documents into chunks, convert them into vector embeddings and store them in a vector database; when a user asks a question, convert the question into a vector as well, retrieve the most relevant chunks via cosine similarity or similar methods, and then feed those chunks along with the question into the model so it can generate an answer grounded in up-to-date content. Compared to stuffing all documents directly into the context window, RAG's advantage lies in its ability to handle large knowledge bases that exceed context window limits, while also significantly reducing model hallucinations when retrieval precision is high. The quality of vector embeddings and the design of retrieval strategies are the key variables that determine how well a RAG system performs.

From Single-Agent to Multi-Agent Collaboration

When a single agent is tasked with too much and its responsibilities become too diffuse, the sensible move is to split the work among multiple specialized agents. This is the origin of multi-agent systems.

In this architecture, an Orchestrator receives the overall task and distributes different parts of it: some goes to a research agent, some to a writing agent, some to a review agent. Each agent plays only one role, equipped with a small set of tools and its own scoped memory.

The essence of this design is isolation. Shared memory has a single owner — the orchestrator. This means that a hallucinated fact produced by one agent won't spread and contaminate all the others; errors are contained locally. Single responsibility and memory isolation are the keys to keeping a multi-agent system reliable.

The Production Contract: MCP

In a real production environment, each agent is an independent service, and MCP (Model Context Protocol) is the contract between them. Through a standardized protocol, different agents can collaborate and communicate reliably without depending on each other's internal implementation details.

MCP (Model Context Protocol) is an open protocol proposed and open-sourced by Anthropic in late 2024, with the goal of standardizing the interface between AI models and external tools and data sources. By analogy, MCP is to AI agents what USB-C is to hardware devices — a single standard interface that lets tools and services from different origins plug in and work immediately, without custom connectors for each model. MCP defines three core primitives: Resources (data for the model to read), Tools (functions for the model to call), and Prompts (prompt templates). In a multi-agent system, each agent exposes its capabilities as an MCP Server, while the orchestrator acts as the MCP Client to centrally coordinate everything. This makes the overall system more modular to extend and replace, avoiding the maintenance nightmare of point-to-point hard coupling between agents.

The Complete Picture at a Glance

String all the concepts together, and the full picture of a multi-agent system is actually quite clear:

• The model is a stateless function;
• An agent is a loop around the model;
• Memory and RAG handle putting the right data into the prompt;
• Multiple agents collaborate, with each one responsible for a single role.

One final piece of practical advice stands out above the rest: start with a single agent. Don't pile on a complex multi-agent architecture from the start. Only add more agents when testing has proven that a single agent genuinely can't handle the job. Engineering restraint is almost always more valuable than architecturally impressive complexity.