Single Agent or Multi-Agent? An Architecture Decision Framework for Enterprise Teams

Starting Point

Teams adopting LLM agents inevitably run into the same recurring question: use a single agent with a well-equipped toolset, or have multiple agents work together? In a demo environment, the multi-agent approach often feels more natural. But in production, it loses its shine quickly — coordination overhead, state bugs, and evaluation burden tend to arrive all at once, and always later than expected.

This article builds a decision framework around the core dimensions that determine whether a multi-agent design is worth the cost: task structure, state management, failure handling, human oversight, latency, cost, reliability, and security. We'll compare LangGraph, CrewAI, and Microsoft AutoGen, as well as Microsoft Agent Framework — the framework Microsoft now recommends for new projects.

Version Note: Microsoft's AutoGen repository is explicitly marked as being in maintenance mode — no new features, community-maintained — and new projects are directed to Microsoft Agent Framework (GA'd at version 1.0 in April 2026). The AutoGen discussion in this article applies only to existing systems; new projects are covered under Agent Framework. All claims reference documentation from October 2026 — verify against the latest docs before shipping.

Single Agent vs Multi-Agent Architecture Comparison

Start with a Single Agent

Build the single-agent version first. An LLM loop with a well-defined toolset forces you to answer the genuinely hard questions early: where are the task boundaries, which tools are actually needed, and where will the agent get stuck.

Stick with a single agent when:

The work is linear. Research then write, fetch then transform. These ordered steps — where each output feeds the next — are pipelines at heart. A single agent calling tools in sequence is simpler and easier to trace than orchestrating multiple agents.
The entire problem fits in one context window. If documents, tool outputs, and history all fit comfortably in context, splitting into multiple agents adds coordination overhead without adding capability.
There's a single accountable owner for the output. Approvals, customer-facing messages, and financial operations all need a clear final decision owner.
The team is new to agents. A single agent teaches you failure modes in full view — tool errors, prompt drift, runaway loops. That experience transfers upward; the reverse is much harder.

When Multi-Agent Is Actually Worth It

None of the following signals are automatically true. Each requires a concrete, testable form. Treat them as hypotheses, not justifications.

Genuinely independent subtasks exist. Parallel research across unrelated sources, or concurrent calls to different systems. The key word is independent: if step B must wait for step A to finish, you have a pipeline — and pipelines don't necessarily need multiple agents.

Roles differ substantially in instructions, tools, or models. A researcher with web search tools and a policy reviewer with compliance documents will undermine each other if crammed into the same system prompt. CrewAI models this directly: each agent carries a role, goal, and backstory; tasks are bound to agents with explicit expected outputs.

Isolation is designed, not assumed. Splitting work across multiple agents doesn't inherently contain failures — a shared downstream consumer still sees every upstream error. Fault isolation must be designed in: per-agent retry budgets, timeouts, and validation at handoff boundaries.

The workflow is long-running and interruptible. Processes spanning hours or days with human checkpoints need persistent, step-by-step state. LangGraph splits this into a checkpointer (thread-level: continuity, human-in-the-loop, time-travel, fault tolerance) and a store (cross-thread facts and preferences). Microsoft Agent Framework's workflow engine similarly provides checkpointing and state recovery.

Different steps have different trust boundaries. When different parts of a workflow run under different credentials or data access scopes, separating agents gives each step a clean identity. But note: different permissions don't always require multiple agents — a single agent can accept scoped credentials per tool call. Multi-agent only pays off here when those boundaries are stable, auditable, and worth enforcing structurally.

When Multi-Agent Doesn't Pay Off

Shared state is where the real complexity lives. Once multiple agents must agree on shared facts — the current plan, customer intent — you have a distributed systems problem. Conflicts and dirty reads don't disappear because the components are language models; managing them is real engineering work.

The debugging surface multiplies. One agent, one trace. Five agents, and you have to reconstruct who said what to whom, in what order, based on which version of shared state. Either budget for observability or stay with a single agent.

Latency compounds on the critical path. Sequential handoffs add model call latency at every step. Count the sequential LLM calls before the user sees output; for interactive scenarios, this alone can disqualify a design.

Evaluation gets harder. Testing agents that interact with each other means covering the interaction space, not just individual outputs — difficult, but not impossible. A workable pattern: unit tests per agent, plus integration tests over the composed workflow. If nobody can articulate how the system will be evaluated, it isn't ready to be built.

Coordination Patterns: A Version-Aware Comparison

Frameworks differ fundamentally in how they handle coordination. Here's a version-aware comparison:

Dimension	LangGraph	CrewAI	AutoGen (existing systems)	Microsoft Agent Framework (new projects)
Status	Active development	Active development	Maintenance mode, no new features, community-maintained	GA at 1.0 in April 2026, Microsoft's recommended successor
Mental model	Explicit graph: nodes are steps, edges are transitions	Role-based teams: agents with roles, goals, and backstories	AgentChat teams (0.4.x): predefined collaboration patterns	Agents + orchestration + graph-based workflows, declarative YAML support
Coordination	Manual wiring: conditional edges, subgraphs	Declarative flows: `sequential` (default) or `hierarchical`	Round-robin, selector, swarm, graph-flow teams	Sequential, concurrent, handoff, group chat, Magentic-One
State management	Typed state schema; checkpointer + store	Task outputs as context; structured `TaskOutput`; guardrail-validated handoffs	Shared context within teams; memory components	Pluggable memory: conversation history, persistent key-value state, vector retrieval
Human-in-the-loop	`interrupt()`: pause anywhere, resume via `Command`	Per-task `human_input` flag	Approval steps within team design	HITL approvals, pause/resume for long-running tasks
Failure recovery	Resume from checkpoint; inspect historical state	`guardrail_max_retries`; manager re-plans in hierarchical mode	Termination conditions constrain execution	Checkpoints; safety/compliance middleware hooks
Best fit	Long-running, stateful, auditable workflows	Expert teams with clear task boundaries	Maintaining existing AutoGen deployments	New enterprise projects, especially on Microsoft stack

One key point: don't mix API generations. The AutoGen 0.2 patterns — GroupChat/GroupChatManager/max_round/TERMINATE — have been superseded, and even the 0.4.x AgentChat API is now in maintenance mode. For new projects, Microsoft points to Agent Framework, which unifies the enterprise foundation of Semantic Kernel with the orchestration research of AutoGen.

Magentic-One is a general-purpose multi-agent system architecture from Microsoft Research, now integrated into Microsoft Agent Framework as a built-in coordination pattern. Its core design features an Orchestrator agent responsible for task planning and progress tracking, working alongside specialized agents (web browsing, code execution, file operations, etc.) to complete open-ended tasks. Unlike CrewAI's static role assignment, Magentic-One's orchestrator dynamically decides at runtime which agent to invoke next, and replans when subtasks fail — making it better suited for scenarios where task structure isn't known in advance, but also harder to predict in terms of execution paths, with higher observability requirements. When using Magentic-One within Agent Framework, its checkpointing and state recovery capabilities are shared with the framework's overall mechanisms, addressing the lack of persistence in the original research implementation.

State: The Decision That Matters Most

Three patterns, distilled:

Explicit state (LangGraph). You declare a schema; nodes read and write named fields. Two separate persistence systems serve distinct purposes: the checkpointer saves thread-level snapshots (pause/resume, time-travel inspection), and the store saves cross-thread facts. The two things teams most commonly get wrong: MemorySaver/InMemorySaver keeps checkpoints in memory, so everything is lost on process restart — production needs a persistent checkpointer. And when resuming from interrupt(), the entire node re-executes from the top — prior code runs again. Side effects before the interrupt must be idempotent, or moved to after it.

Validated handoffs (CrewAI). Task outputs flow downstream as context, wrapped in a structured TaskOutput (raw, JSON, or Pydantic-validated). The correction to the "no inspectable state" stereotype is guardrails: tasks can carry function-based or LLM-based guardrails that validate output before the next task runs, with configurable retries (guardrail_max_retries) — a concrete mechanism for enforcing state invariants at handoff boundaries.

Conversation as state (team group chat architecture). Agents coordinate through shared conversational context — highly expressive, but hardest to reason about. State invariants can still be enforced, just at higher cost: validation steps, structured outputs, and termination conditions that halt the team on violations.

Whichever pattern you choose, write down the invariants first: which facts must never go stale, who can modify them, and what will cause the system to stop if there's a conflict.

The underlying mechanics of checkpointing and persistence are worth a brief explanation. LangGraph's checkpointer is fundamentally a snapshot mechanism: whenever graph execution reaches a node boundary, the current complete state dictionary is serialized and written to a storage backend (in-memory, SQLite, PostgreSQL, etc.). This means that even if a process crashes, the next resumption can replay from the last successful checkpoint rather than re-running the entire workflow from scratch. "Time-travel" debugging exploits exactly this mechanism: developers can load any historical snapshot, modify inputs, and re-execute forward from that node — useful for retroactive questions like "what went wrong at step three." The cost of this mechanism is storage and serialization overhead, plus the requirement that node-level side effects be idempotent. Irreversible operations — writing to external APIs, sending emails — will be re-run on recovery if they happened before an interrupt, so they should be deferred to after the interrupt() call or protected with idempotency guards.

Failure Handling Is Designed, Not Inherited

Assume every component will fail; the design question is how large the blast radius is:

Tool failures (API outages, malformed output): per-tool retries and fallbacks — never let one tool's failure mode become the entire system's failure mode.
Delegation loops (A asks B, B asks A back): constrain with termination conditions, round limits, and watchdog timeouts.
State corruption: a bad write poisons downstream work. Defend with validation at write boundaries and checkpoint inspection before recovering failed runs.
Silent quality degradation: the system runs, but produces mediocre output. No framework solves this; it requires an evaluation suite with real labeled samples, run on every change.

Human Oversight: Where Should Humans Sit?

Enterprise scenarios make this non-optional, and frameworks differ in granularity:

LangGraph: interrupt() can pause anywhere — inside a node, even inside a tool — state is persisted via checkpointer and resumed via Command. Static interrupt_before/interrupt_after covers known checkpoints. Remember the re-execution rule: keep pre-interrupt code side-effect-free or idempotent.
CrewAI: the human_input flag places human review at task boundaries — coarser granularity, but easier to reason about.
Microsoft Agent Framework: human-in-the-loop approvals with pause/resume, plus middleware hooks for compliance policies without touching prompts.

Place gates where the cost of being wrong exceeds the cost of waiting: production writes, customer-facing sends, financial actions. Drafting an internal summary doesn't warrant an approval gate.

Latency, Cost, Reliability, Security: Do the Math

Latency: count the sequential LLM calls on the critical path and multiply by typical per-call latency. If the total exceeds your UX budget, redesign before building.

Cost: (calls per run) × (tokens per call) × (runs per day) × (price per token). Multi-agent designs amplify the first three factors; run these numbers at production scale, not demo scale.

Reliability: the textbook estimate — ten sequential steps each at 95% reliability yields roughly 60% end-to-end (0.95^10 ≈ 59.9%) — rests on a simplified independence assumption that failures are uncorrelated. Correlation shifts this in either direction: shared failure causes make joint failures more likely, while shared mitigations (unified validation, retries) make them less so. Use this number as a starting point for "add validation, measure your own step reliability" — not as a prediction.

Security: more agents means more trust boundaries. Scope credentials per agent or per tool call; treat inter-agent messages as untrusted input across trust layers (prompt injection propagates through agent teams just as it does across microservices); log decisions for audit. An over-privileged agent is a single point of risk; five credential sets are five places things can be misconfigured.

A small pilot that measures a single-agent baseline on latency, cost, and reliability teaches you more than any architecture diagram.

The propagation mechanics of prompt injection in multi-agent environments deserve special attention. In a single-agent system, the attack surface is primarily user input and tool return values. In a multi-agent system, Agent A's output becomes Agent B's input — an attacker only needs to contaminate one upstream node's output, and malicious instructions can propagate through the entire team via message passing. This is structurally identical to injection attacks spreading through intermediate data stores in microservices. Defenses include: treating inter-agent messages as untrusted input and performing structured validation (rather than splicing them directly into prompts); sandboxing content from external data sources (web pages, files, databases); and placing human approval gates before high-risk tool calls. The principle of least-privilege credentials also serves double duty here: even if an attacker successfully injects malicious instructions, scoped credentials limit the actual operations that can be executed, reducing the blast radius.

An Example and a Pre-Launch Checklist

Imagine a support ticket triage workflow: tickets are classified, enriched with customer history, then routed, with human approval required for routing decisions that change SLA.

Single-agent version: one agent, three tools (classifier, CRM lookup, router), with an approval gate before the router tool. Every decision is visible in a single record.
Multi-agent version: separate triage, enrichment, and routing agents — only worth considering when concrete signals emerge: slow independent enrichment calls (parallelism), or routing rules that change rapidly enough to warrant separate ownership and an evaluation suite (specialization, with fault isolation explicitly designed in).

"Feels more scalable" is not a signal. A measured baseline is.

Before going live, work through this checklist:

Single-agent baseline has been built and measured (latency, cost, reliability)
At least one multi-agent signal has been concretely validated as true, not assumed
State invariants are written down: what must never go stale, who can modify it, what will halt the system on conflict
Persistence choices are explicit: what survives a restart and what doesn't
Framework choice is version-aware: don't build new enterprise systems on a maintenance-mode framework without a migration plan
Human approval gates are placed where the cost of being wrong exceeds the cost of waiting
Termination conditions, round limits, retry budgets, and idempotency rules are all defined
An evaluation suite with real labeled samples runs on every change
Credential scoping and audit logging are designed in, not deferred
The team can explain system behavior from a trace, not from an architecture diagram

Disclosure: This article was written with AI assistance. Technical claims have been verified against official framework documentation from October 2026 — validate against the latest docs before shipping.