Harness Multi-Agent Framework: A Deep Dive into Planner→Builder→Evaluator Three-Agent Collaboration

From Interview Questions to AI Agent Technology Trends

In the wave of productionizing large model applications, Multi-Agent architectures are transitioning from concept to enterprise-level practice. The concept of multi-agent architecture originates from distributed artificial intelligence, traceable back to distributed problem-solving research in the 1980s. However, in the era of large language models, Multi-Agent has taken on an entirely new meaning: multiple LLM-driven agents, each with specialized roles, collaborating to complete complex tasks. Since 2023, projects like Stanford's Generative Agents, MetaGPT, and AutoGen have validated the feasibility of multi-agent collaboration. Entering 2024-2025, enterprises have begun moving this paradigm from the lab into production environments, driving strong demand for standardized frameworks and engineering capabilities.

An instructor specializing in LLM interview coaching revealed during a livestream that since launching their LLM course, the team has coached over 1,700 students, revised 1,065 resumes, and compiled 625 interview recordings with post-mortems. This firsthand data reveals a clear trend: Harness (the agent runtime framework) is becoming a core topic in AI job interviews.

Based on interview questions reported by students, the questions are highly concentrated around Harness-related technical details.

Topics that interviewers repeatedly probe include: multi-agent collaboration, Agent Loop implementation mechanisms, distributed deployment of agents, error handling for tool invocations, solutions for circular invocation problems, and the critically important user isolation and security isolation (Sandbox). These questions almost invariably point to the underlying design capabilities of the Harness framework.

Why Harness Has Become a Technical Hot Topic

Harness has become an interview hot topic due to its close ties with recent industry developments. With the release of DeepSeek's Harness public beta, numerous tech bloggers and frontline developers have conducted hands-on evaluations. The instructor noted that DeepSeek Harness excels in flexibility — thanks to its support for freely installing plugins, it is in some ways even more flexible than Claude Code and Codex, while being no less capable in terms of functionality.

The direct impact of this trend is that more and more enterprises are considering building their own agent systems on top of Harness. For developers, this means that mastering the internal implementation principles of Harness is not just an interview bonus — it's a hard requirement for production-level engineering.

Planner→Builder→Evaluator: The Core Paradigm of Three-Agent Collaboration

In multi-agent architectures, a classic and highly effective collaboration pattern is the Planner→Builder→Evaluator three-stage pipeline. Understanding this paradigm is the foundation for implementing a Harness multi-agent framework from scratch. This pattern draws from the classic "design-implement-test" workflow in software engineering, mapping it onto automated agent collaboration.

Planner: Task Understanding and Decomposition

The Planner is responsible for receiving the user's original requirements and breaking them down into an executable sequence of subtasks. Its core responsibilities are "intent understanding" and "task decomposition." An effective Planner must determine which tasks can run in parallel and which have dependencies, thereby constructing a reasonable execution plan (DAG — Directed Acyclic Graph).

DAG is a fundamental concept in graph theory, referring to a directed graph without cycles. In the task orchestration domain, DAGs are widely used to describe dependencies between tasks — nodes represent subtasks, and directed edges represent execution order constraints. Workflow orchestration tools like Apache Airflow and Prefect use DAGs as their core abstraction. In multi-agent systems, when the Planner decomposes user requirements into a DAG structure, it means the system can identify which subtasks have sequential dependencies (must execute serially) and which are independent (can be scheduled in parallel), thereby maximizing execution efficiency and avoiding resource waste.

Builder: Tool Invocation and Task Execution

The Builder takes the task list output by the Planner and sequentially invokes tools, writes code, or calls other sub-agents to complete specific work. At this stage, accuracy of tool invocation becomes a key challenge. When a system integrates a large number of tools, preventing agents from calling the wrong tool is a high-frequency interview question. Common solutions include:

• Semantic optimization of tool descriptions
• Schema-based parameter validation
• Introducing a Tool Routing mechanism to narrow down candidate options

Tool Routing borrows from information retrieval and semantic matching approaches: first, embedding vector retrieval or classifiers are used to pre-judge user intent, narrowing the candidate tool set to 3-5 options, which are then passed to the LLM for precise selection. This is analogous to the recall-ranking two-stage architecture used in search engines, significantly improving tool invocation accuracy while reducing latency. Some frameworks also introduce the concept of Tool Namespace — logically partitioning tools by business domain to further reduce the complexity of the selection space.

Evaluator: Quality Validation and Closed-Loop Feedback

The Evaluator performs quality validation and result feedback on the Builder's output. If results don't meet expectations, it sends information back to the Planner or Builder, forming a closed-loop Agent Loop. This looping mechanism is both the core of how multi-agent systems achieve self-correction and a source of new engineering risks — the circular invocation problem.

Key Technical Challenges: Circular Invocation Prevention and Security Isolation

How to Avoid Circular Invocations in the Agent Loop

In an Agent Loop, if the Evaluator continuously judges results as unsatisfactory, the system can fall into an infinite loop, causing token consumption to skyrocket and response times to spiral out of control.

To understand the severity of this issue, you need to recognize what tokens mean in terms of cost for LLM applications. In large model applications, tokens serve as both a unit of computational resources and a direct cost driver. Taking GPT-4o as an example, input tokens cost approximately $2.5 per million tokens, while output tokens cost around $10. In a multi-agent system's Agent Loop, each iteration means at least one complete LLM inference call — encompassing system prompts, historical context, tool descriptions, and the full token payload for the current task. If the loop runs uncontrolled for more than 10 iterations, the cost of a single request can surge by tens of times. Therefore, Token Budget management has become a standard feature of production-grade Agent systems, typically including context window compression, conversation history summarization, and context strategies that decrease with each iteration round.

Engineering teams typically mitigate circular invocation issues through the following strategies:

• Setting a hard cap on the maximum number of iterations
• Introducing loop detection mechanisms (such as state hash comparison)
• Defining explicit convergence conditions for each iteration

Controlling token costs and reducing response times are key metrics for evaluating the maturity of a multi-agent system.

Sandbox Security Isolation and Multi-Tenant User Isolation

The Sandbox security isolation repeatedly emphasized by interviewers is essentially about solving resource and data isolation problems in multi-user, multi-tenant scenarios. When agents need to execute user-submitted code or access sensitive resources, a sandbox environment must be used to limit their permission boundaries, preventing unauthorized access and data leaks. This is also the core differentiator between enterprise-grade Harness frameworks and toy-level demos.

In engineering implementation, Sandbox has multiple technical approaches. The most common is using containerization technologies (such as Docker, gVisor) to create independent runtime environments for each user or code execution, restricting CPU, memory, network, and filesystem access through namespaces and cgroups. Lighter-weight alternatives include WebAssembly (Wasm) sandboxes, which achieve memory-safe isolation at near-native performance and have been widely adopted in edge computing and plugin systems in recent years. For multi-tenant SaaS scenarios, tenant isolation at the data layer is also required — typically implemented through independent database schemas, Row-Level Security (RLS) policies, or encryption-based isolation to ensure data invisibility between different users. The choice of isolation granularity depends on the trade-off between security requirements and performance overhead.

A2A Protocol vs. SubAgent: Selection Strategy for Internal Agent Handoff

Regarding communication methods between agents, the instructor offered a pragmatic assessment: The A2A (Agent-to-Agent) protocol is not widely used within enterprises; the SubAgent invocation approach is far more common.

The A2A (Agent-to-Agent) protocol was officially released by Google in April 2025, aiming to establish an open standard for cross-platform agent interoperability. Its core mechanisms include Agent Card (an agent's business card in JSON format describing capabilities, endpoints, and authentication methods), Task objects (defining the lifecycle and state machine of interactions), and a communication layer based on HTTP+SSE/JSON-RPC. A2A's design philosophy is similar to service discovery and API gateways in microservice architectures — establishing a unified interaction contract between heterogeneous systems. It complements MCP (Model Context Protocol): MCP addresses the connection between models and tools, while A2A addresses collaboration between agents.

The reason is that A2A is inherently designed for cross-network, cross-process communication. It's suited for handoffs and deliveries between agents on different platforms, from different teams, or even built with different technology stacks. In such "no shared context" scenarios, A2A is practically the only option.

However, if the entire multi-agent system is designed by the same team — say, containing seven or eight sub-agents internally — then using A2A within the system is like "using a cannon to kill a mosquito." Cross-network transmission, card discovery (Agent Card), card reading, and other steps all consume extra time. The instructor provided an intuitive quantitative comparison: A well-designed internal multi-agent system should respond within 1 second, but introducing A2A pushes that to at least 2 seconds, noticeably degrading user experience.

By contrast, SubAgent invocation is an agent orchestration approach within the same process or runtime environment. Unlike A2A's cross-network communication, SubAgents pass data directly through function calls, message queues, or shared memory, eliminating the overhead of HTTP serialization/deserialization, network latency, and authentication. In mainstream frameworks like OpenAI's Agents SDK and LangGraph, SubAgents are typically modeled as Tools or Nodes that can be invoked by a parent Agent, with their execution context managed uniformly by the orchestration layer. The advantage of this pattern is low latency and high controllability; the disadvantage is tighter coupling, making it unsuitable for cross-team or cross-organization agent collaboration. Therefore, internal agent handoffs are better suited to lightweight approaches like SubAgent.

Practical Harness Recommendations for Job Seekers and Engineers

Given current interview trends and technological evolution, mastering the Harness framework should be approached from the following dimensions:

• Understand the Three-Agent Collaboration Paradigm: Be able to clearly articulate the responsibilities and collaboration workflow of Planner, Builder, and Evaluator, and understand how they map to the design-implement-test workflow in traditional software engineering
• Master Agent Loop Closed-Loop Design: Including feedback mechanisms, convergence conditions, and loop prevention, while also understanding how Token Budget management impacts sustainable system operation
• Focus on Engineering Details: Tool invocation error prevention, token cost optimization, and response time control — especially the necessity of Tool Routing mechanisms in large-scale tool integration scenarios
• Go Deep on Security Isolation: Understand the necessity and implementation approaches of Sandbox in multi-tenant scenarios, and master the applicable scenarios for different granularity solutions including containerized isolation, Wasm sandboxes, and data-layer RLS
• Differentiate Communication Protocols: Clearly understand the applicable boundaries of A2A and SubAgent to avoid blind adoption — A2A is for cross-organization heterogeneous collaboration, SubAgent is for efficient intra-team orchestration

With the maturation of open-source frameworks like DeepSeek Harness, building a multi-agent collaboration framework from scratch is no longer an unattainable goal. For developers looking to establish themselves in the AI engineering field, deeply understanding the internal mechanisms of Harness will be one of the most valuable technical investments they can make.