Harness Architecture in Practice: A Deep Dive into Enterprise-Level Agent Projects

Why Harness Architecture Has Become a New Focus in LLM Interviews

The bar for landing a job in the AI/LLM space is rising fast. As shared by instructor Xiao from Mashibing Education on Bilibili, interviewers across both algorithm research and LLM application development roles are demanding greater technical depth. One notable shift: interviewers are now placing heavy emphasis on candidates' understanding of Multi-Agent systems and Harness architecture.

Harness (short for "harnessing engineering") is a next-generation agent architecture proposed by Anthropic for handling complex tasks. Rather than simply "feeding a prompt to a model," it establishes a comprehensive engineering system around intelligent agents — covering model scheduling, tool invocation, security isolation, and data integration. Worth noting: Harness architecture grew out of Anthropic's 2024 agentic engineering practice framework, and its core philosophy is to elevate single LLM calls into an orchestrable, observable, and scalable engineering system. Compared to the earlier ReAct (Reasoning + Acting) paradigm — where models alternated between "reasoning" and "acting" to complete tasks — Harness goes further by bringing the entire agent lifecycle under unified engineering control, prioritizing reliability and safety in production environments rather than merely improving single-turn task quality. This article uses a real-world enterprise project to break down the core design of this architecture layer by layer.

Two Career Tracks in LLM Roles

Before diving into the architecture, it's worth clarifying the two main career directions in the LLM space:

Algorithm Research Track: Focused on developing new foundation models and participating in model pre-training. These roles have high academic and publication requirements, typically requiring a master's degree or above from a top-tier university.
LLM Engineering & Deployment Track: Customizing existing foundation models for enterprise business needs, involving fine-tuning, Agent development, inference optimization, and more — generally without involvement in pre-training.

LLM role classification by track

Interestingly, job titles in the engineering track vary widely — Application Engineer, LLM Application Algorithm Engineer, Agent Development Engineer, and more. The title doesn't matter much; what's important is understanding the actual work behind the role.

Enterprise Agent Project Overview: An Intelligent Procurement Assistant

The project analyzed in this article is an intelligent procurement assistant built on top of an existing ERP system — a textbook Harness architecture case study. The emphasis on "built on top of an ERP system" reflects the reality of enterprise agent development: most companies were already running mature business systems like CRM and ERP long before LLMs became mainstream. New agents can't exist in isolation — they must deeply integrate with these existing systems.

Front-End/Back-End Separated Architecture

The project uses a front-end/back-end separation design:

Front end: Built with Vue, running on port 3000, handling the user interaction interface.
Back end: Built with the FastAPI framework, running on port 8090, with the agent ultimately deployed in an ASGI service.

Overall project architecture diagram

Why must agents in enterprise production environments be deployed to an ASGI service? To understand this, some background on ASGI is needed. ASGI (Asynchronous Server Gateway Interface) is the async successor to WSGI in the Python web ecosystem, first popularized by the Django Channels project in 2018. Unlike traditional WSGI (a synchronous blocking model), ASGI is built on Python's asyncio event loop and can handle thousands of concurrent connections in a single process — critical for scenarios where large numbers of users interact with an agent simultaneously. In this project, there are two core reasons for choosing ASGI:

High-concurrency async processing: ASGI natively supports asynchronous requests, handling large numbers of concurrent users gracefully. While one user's request is waiting for a response from the LLM, the server doesn't block — it can simultaneously handle other users' requests.
True streaming output: Streaming responses are key to a smooth agent interaction experience — users see the model's output word by word rather than waiting for a complete answer. ASGI's async nature natively supports this capability, whereas traditional synchronous frameworks require considerable engineering effort to approximate it.

The project uses UV as the ASGI runtime framework. The combination of "Agent + ASGI backend + front end" is the standard setup for enterprise production environments — not the simplified approach of a personal side project.

MCP Protocol: The Standard Solution for Connecting Agents to Enterprise Systems

One of the core capabilities of the intelligent procurement assistant is reading real business data from the company's ERP system. The project uses the MCP (Model Context Protocol) protocol and gateway to enable standardized access to ERP data from an existing Java project.

To understand why MCP matters, it helps to know what problem it solves. MCP is a standardized protocol open-sourced by Anthropic in November 2024. Its origin lies in the "fragmentation problem" of integrating LLMs with external systems: before MCP, every AI application had to write custom adapter code for each data source — one set of logic for databases, another for file systems, another for REST APIs — resulting in high maintenance costs and poor reusability. MCP provides a unified client-server protocol that allows models to access file systems, databases, APIs, and any other resources in a standardized way, much like how USB standardized hardware connectivity. This design addresses the universal challenge of "how agents can safely and consistently access enterprise internal data."

Reading ERP project data via MCP protocol and gateway

At the model layer, the project integrates multiple external models with clearly defined roles:

Reasoning model: DeepSeek serves as the primary reasoning engine
Fallback model: Zhipu AI takes over when the primary model is unavailable
Summarization model: Configured separately, dedicated to text summarization tasks

This multi-model collaboration design is a hallmark of Harness architecture — letting the most suitable model handle each type of task, rather than relying on a single model to handle everything. This concept is known as "Model Routing" in engineering terms. The underlying logic is that reasoning tasks require a strong chain of logic, while summarization tasks demand language concision — the two emphasize different model capabilities. Forcing a single model to handle both wastes compute and may not yield the best results.

Fetching External Data: Connecting to Supplier Information

As a procurement assistant, the agent also needs real-time supplier quotes. The project accesses supplier websites via Skill invocations or web crawlers, scraping pricing pages and product information to provide external data for procurement decisions.

Sandbox Mechanism: The Security Foundation for Enterprise-Grade Agents

While MCP addresses data integration, the Sandbox handles two security concerns that are easily overlooked yet critically important in enterprise-grade agent deployments.

Secure Isolation for Skill Execution

The agent supports Skills (capability extensions) that may be downloaded from the internet or even generated autonomously by the agent itself. Regardless of their source, executing them directly in the host environment poses a security risk — especially auto-generated code, which may contain dangerous operations or malicious logic. This risk is particularly acute when an LLM has code generation and autonomous execution capabilities: the model could be manipulated into generating code that deletes files, accesses sensitive directories, or even launches network attacks. Without isolation, the consequences are unpredictable.

Data Isolation Across Multiple Users

The front end serves multiple users, and the agent generates large volumes of intermediate files, analysis reports, and other temporary data when processing complex tasks.

Agent-generated files illustration

Without user isolation, files from different users can easily become mixed up or leaked. The solution is to allocate an independent sandbox to each user: Zhang San's sandbox belongs only to Zhang San, Li Si's sandbox belongs only to Li Si — data isolation is achieved architecturally by design.

The project deploys OpenSandbox-Server on a dedicated Linux server to provide sandbox services. Technically, sandboxes are implemented using container technology — a mature capability pioneered by Docker in 2013 that has since become a core component of cloud-native infrastructure. Containers leverage Linux kernel mechanisms — namespaces and cgroups — to provide each running instance with its own isolated file system, network stack, and process space. This allows different users' agent tasks to physically share the same server while remaining logically completely isolated from one another. To support the varied runtime environments different users may need, the project also configures an image registry with multiple pre-built container images that can be dynamically allocated on demand — a philosophy consistent with Kubernetes production cluster management.

Full Capability Map of Harness Architecture and Career Insights

Taken together, this intelligent procurement assistant project integrates multiple core technologies essential to enterprise-grade Agent development:

Technology Module	Core Role
Harness Architecture	Engineering orchestration framework for complex tasks
Multi-Model Scheduling	Reasoning, fallback, and summarization models each with defined responsibilities
MCP Protocol	Standardized integration with existing enterprise business systems
ASGI Deployment	Supports high-concurrency requests and streaming output
Sandbox Isolation	Ensures code safety and data isolation across multiple users

For job seekers preparing for LLM roles, this architecture maps directly onto what interviewers are now looking for. Simply knowing how to call an API or write prompts is no longer enough — the real competitive edge today is the ability to understand and implement a complete Agent engineering system that mirrors a genuine enterprise production environment. This means candidates must simultaneously develop an "AI perspective" (understanding model capability boundaries and multi-model collaboration) and an "engineering perspective" (mastering async services, container isolation, protocol standardization, and other infrastructure fundamentals) — neither alone is sufficient.

This article has outlined the high-level architectural framework of the project. Specific code structure and detailed implementation will be covered in subsequent content.