Agent Harness Deep Dive: From Framework Comparison to Core Component Implementation

In AI Agent development, we often hear the terms "framework" and "Harness." They may seem similar, but they actually represent two completely different engineering philosophies. Based on a technical talk on Bilibili, this article uses two open-source projects, DeepFlow and DeepSeek Harness, as examples to break down the architecture design and implementation details of Agent Harness.

What Is a Harness? How Is It Different from an Agent Framework?

Harness originally means horse tack, used to control a horse's behavior. In the Agent context, it refers to the operational layer that wraps the model and provides a runtime environment. A common definition is: Agent = Model + Harness, meaning the Harness is everything in an Agent besides the model.

Compared with Agent Frameworks like LangChain, a Framework is more like a programming library—it provides abstractions over LLMs, tool primitives, memory, and orchestration capabilities, but you need to implement the tool-calling loop, session management, network retries, and other details yourself. A Harness, by contrast, is an out-of-the-box, complete execution environment—essentially an already-written Agent.

Why do we need Harness when we already have Frameworks? There are two core reasons: first, to avoid reinventing the wheel—the underlying logic of general-purpose Agents is highly similar, and a standardized Harness can be reused; second, models and Harnesses co-evolve—the same model running on different Harnesses on the Terminal Bench dataset produces different results. Models released by vendors are often tuned for specific Harnesses, and a self-built Harness may not fully unlock the model's capabilities.

The Three Core Components of a Harness

From an architectural perspective, a Harness can be summarized into three parts:

1. Runtime (model runtime): Essentially the model's operating system, responsible for executing tool calls and managing processes and sandbox environments.
2. User Interface: Allows users to send messages, control Agent permissions, adjust thinking depth, and view the execution process.
3. Context Manager: Influences model behavior in reverse by organizing and modifying Context, including System Prompt, memory injection, tool result trimming, and more.

Together, these three form the complete closed loop of a Harness: the Runtime lets the model "do things," the UI lets the user "stay in control," and the Context Manager lets the model "know what to do."

Component 1: Orchestration Loop

Almost all general-purpose Agents use the ReAct pattern: Thought (model reasoning) → Action (tool call) → Observation (tool result), repeating in a loop. Many Harness components are inserted at various stages of this loop.

DeepSeek^Ad Harness hand-writes a three-layer loop in TypeScript: the outer layer is the multi-turn conversation between the Agent and the user; the middle layer is the Thought-Action-Observation loop; the inner layer is the network retry when a model call fails.

DeepFlow implements a standard Model-Tools loop based on LangGraph and inserts extension points into the loop through a Middleware mechanism. For example, Before Agent Middleware organizes static content such as the current date and System Prompt before the Agent starts, ensuring the Prefix Context stays fixed and thus saving Tokens. Before Model Middleware can include summarization and compression logic to detect whether the context is approaching the model's limit.

The ReAct (Reasoning + Acting) pattern was proposed by Yao et al. in 2022. Its core idea is to have the model alternate between generating reasoning traces and action instructions, thereby making "thinking" explicit and compensating for the inability of pure chain-of-thought reasoning to interact with the external environment. Compared with the earlier Chain-of-Thought, which can only produce internal reasoning text, ReAct's key contribution is the introduction of the Observation stage: the real return values from tools are fed back into the context, and the model uses them to adjust its next plan, forming a closed feedback loop. This is the fundamental reason it has become the de facto standard for general-purpose Agents—the model does not need to plan the entire path at once and can act while observing and correct errors dynamically. The cost is that each round of tool calls consumes extra Tokens and latency, and erroneous observations may contaminate subsequent reasoning. Therefore, in long tasks it must be combined with loop detection, budget control, and context compression to avoid falling into repeated calls or context explosion.

Component 2: Tools

Tools are the Agent's hands. Tools are usually defined in Schema form, including name, functional description, and parameters. Common tool categories include: sub-Agent systems, file operations, search, Shell command execution, Web access, task planning (To-Do List), Skill invocation, and IDE-specific tools.

One implementation detail worth noting is the To-Do List tool. The implementation in DeepSeek Harness is stateless—the Harness does not record task progress and only checks two constraints: all task states must be one of pending, in-progress, or done; and only one task can be in progress at a time. As for adding or removing tasks and tracking completion, the model maintains all of that itself.

Tool calling relies on model providers' Guided Decoding technology. After the model outputs the special token for a tool call, the Provider forces the next Token to be a JSON curly brace and sets the logits of other Tokens at enumeration fields to negative infinity, allowing only valid options to participate in sampling. This is how the Harness can reliably parse tool calls.

Guided Decoding is a constrained sampling technique on the inference server side. Common implementations include grammar constraints based on finite state machines and structured output based on JSON Schema. The principle is to compute the set of valid Tokens at each decoding step based on the generated prefix, and push the logits outside these positions to negative infinity so that sampling can only occur within the valid set. For the Harness, this means tool call parameters can be guaranteed to be valid JSON and satisfy type constraints such as enums and required fields, avoiding parsing failures and retry overhead. The limitation is that the more complex the constraints, the higher the cost of maintaining prefix states, and strong constraints may interfere with the model's original probability distribution, slightly reducing semantic quality when parameters are long or deeply nested.

Component 3: File System

The file system solves three problems: breaking through Context capacity limits, providing persistent memory, and serving as a shared intermediate layer for multiple Agents. The rationale for letting models use the human file system directly is that models have already seen a large amount of Shell and file operation code during training, so there is no need to design a dedicated Agent file system.

DeepFlow provides two modes: Docker and Local Sandbox. In Local mode, path virtualization is implemented: the System Prompt tells the model that user files are located at virtual paths, and the Harness performs forward and reverse path mapping during tool calls.

However, this design "stops honest people but not malicious ones"—the model can bypass virtual paths through code concatenation. Therefore, DeepFlow enables Docker mode by default, while Local Sandbox must be actively enabled and Bash is disabled by default.

Components 4–6: Bash Execution, Sandbox, and Memory

Bash and code execution allow the Agent to write its own tools, compensating for the shortcomings of predefined tools. Claude Code manages Bash commands through a blacklist/whitelist mechanism: first check the deny list, then the allow list, then the read-only whitelist, and finally ask the user.

Sandbox is responsible for security isolation, both preventing access to sensitive files and supporting multi-tenant isolation. DeepSeek Harness uses a kernel-level sandbox, prepending sandbox instructions before executing Python scripts and enforcing access control on processes at the kernel level.

Memory's core goal is Continuous Learning Without Re-Changing. It is divided into long-term memory (such as user preferences and new knowledge in Agent.md) and short-term memory (current task progress). DeepFlow's Memory mechanism is an After Agent Middleware. After each round of conversation, it goes through three filtering steps: keep only the user's initial input and the AI's final output; filter out pleasantry turns; detect user calibration or preference signals. Then it calls an independent model to judge whether the information is persistent and descriptive and outputs a confidence score; only information that meets the threshold is written to Agent.md.

Kernel-level sandboxes usually rely on Linux namespaces and cgroups for isolation: the former isolates the process view, network, and mount points, while the latter limits CPU, memory, and process count. Finer-grained file access control can use seccomp to filter system calls, or mechanisms such as Landlock and gVisor to narrow process permissions. Unlike containers that share the kernel, these solutions impose policies at process startup, so even if Agent-generated code has privilege-escalation intent, it is difficult to break through the restrictions. The cost is compatibility—some scripts that depend on special system calls may fail to run, requiring a trade-off between isolation strength and usability. This is also why Agent sandboxes generally provide multiple security levels.

Component 7: Context Management

Context does not have to be Append Only. Although append mode is the most KV Cache-friendly, it is not necessarily optimal for task execution. The static part is mainly the System Prompt. DeepFlow's System Prompt includes: Soul (Agent persona), Thinking Style (think clearly before acting), Skill System, SubAgent System (a reminder not to over-delegate), and Critical Reminder (deliverable paths, file modification interface preferences, etc.).

The dynamic part consists of Context Management behaviors such as Memory and tool result injection, which together shape the "world" the model sees in each round.

KV Cache is a GPU memory optimization for autoregressive inference: it caches the Key/Value tensors of processed Tokens so that subsequent generation only needs to compute attention for new Tokens, reducing the per-step complexity from quadratic in sequence length to linear. When the prefix is exactly the same, the cache can be hit directly, resulting in faster responses and lower cost; once content is inserted or modified in the middle, all subsequent caches become invalid and must be recomputed. This explains why Append Only context is the most inference-friendly and why DeepFlow fixes static prefixes such as the System Prompt before the Agent starts and injects memory and tool results afterward. Operations such as compression and reordering improve task performance, but each triggers cache rebuilding, which is a cost that must be weighed.

Component 8: Context Rot Defense

Context is a scarce resource; exceeding the model window degrades reasoning quality. There are three common defense methods: Compaction (summarize and compress history when approaching the limit), Offloading (temporarily store tool output in the file system instead of stuffing it all back into Context), and Skill (three-level progressive disclosure, with only descriptions in the System Prompt and Skill.md and deeper files loaded on demand).

Compaction is triggered by Summarization Middleware at the Before Model position, with trigger conditions including conversation turns or Token thresholds. Token estimation commonly uses the formula of character count divided by 4, which is accurate for English but underestimates Chinese and code.

Component 9: Long Horizon Execution

Complex tasks spanning multiple reasoning steps and Contexts face three major challenges: forgetting caused by overly long context, repeatedly calling tools on the wrong path, and unclear goal decomposition. The solution combines file system Offload, To-Do List planning, and Self Verification.

DeepFlow's Go Loop mechanism inserts a "strict completion judge" model at the After Agent position to determine whether the current return truly satisfies the user's goal. If not completed, the Agent is asked to continue, with the prompt "Do not ask the user to continue unless you are truly blocked." The loop runs for at most eight rounds, with a sliding window to prevent infinite loops.

Components 10–12: Error Handling, Serving, and Observability

Error handling covers loop detection and Token budget control. Loop Detection hashes the type of tool call and inserts a warning message when a threshold is exceeded; Token Budget Middleware reminds the model to output existing results as soon as possible when usage exceeds the limit.

The Serving Layer abstracts platform channels into a unified interface. DeepFlow connects to different platforms through Channel Adapter; for example, WeChat iLink requires active message polling, and files are transferred via CDN.

In terms of Observability, DeepSeek Harness's Session Log design is quite inspiring: it is the Append Only single source of truth, and both frontend rendering and Context Manager work by reading the Session Log. The advantage is that execution paths can be deterministically reproduced and bugs are easier to locate; the disadvantage is the enormous log volume, and once the Projection protocol is fixed, subsequent improvements may face compatibility issues.

Co-evolution of Models and Harnesses

DeepSeek once released a model trained together with DeepSeek Harness, which is a clear trend. The benefit is that the model is adapted to the Harness and performs better; tools repeatedly used during model training can also be incorporated into the Harness. The negative effect is overfitting—the model may not transfer to other Harnesses, and generalization ability declines. A report from OpenAI Codex once pointed out that training data was overly concentrated on Apply Patch as the only way to modify files, causing the model's ability in other methods to degrade.

The essence of a Harness is to provide a standardized, reusable runtime environment for the model while shaping model behavior in reverse through Context management. Understanding its component design helps developers build reliable Agent systems more than simply calling a framework.

One-line Summary

Breaks down the definition, three core components, and twelve key components of Agent Harness, and distinguishes it from frameworks.

Paragraph Summary

The article starts with conceptual clarification, pointing out that the Harness is everything in an Agent besides the model, and unlike Frameworks such as LangChain that provide abstract primitives, it is an out-of-the-box complete execution environment. A Harness consists of three core parts: Runtime, User Interface, and Context Manager, and can be further divided into twelve types of components, including orchestration loop, tools, file system, Bash and sandbox, memory, context management, context rot defense, long-horizon execution, error handling, Serving, and observability. Using DeepFlow and DeepSeek Harness as examples, the article compares two implementation paths—Middleware extension mechanism and hand-written three-layer loop—and discusses the benefits and overfitting risks brought by the co-evolution of models and Harnesses.