Harness Engineering in Practice: A Deep Dive into Enterprise Multi-Agent Architecture

What Is Harness Engineering Architecture

As large model agents move from concept to production, building reliable and scalable Agent systems within enterprise environments has become a central engineering challenge. A technical creator on Bilibili shared a real-world client project — an intelligent procurement assistant built on top of an existing ERP system — using an architectural approach known as "Harness Engineering." This architecture treats agents as integral components of an engineering system rather than isolated AI capability modules.

The core philosophy of Harness Engineering is this: agents are not standalone black boxes. They must deeply collaborate with external models, existing business systems, execution environments, and user interfaces. The project itself is substantial in codebase size — a complete, enterprise-grade hands-on case study that covers the full stack from model integration to production deployment.

A Frontend/Backend Separated Architecture

The project follows the classic frontend/backend separation pattern. The frontend is built with Vue, running on port 3000, handling user interaction. The backend is built with FastAPI, running on port 8090. A dedicated backend is necessary because any agent developed for production must ultimately be deployed into a production-grade service.

For Python-based agents, the backend typically runs on an ASGI (Asynchronous Server Gateway Interface) server. The project uses the UV toolchain as the ASGI runtime. The creator emphasizes that ASGI has become the standard for enterprise production environments primarily because it solves two critical problems:

• High-concurrency async handling: Capable of managing high-frequency asynchronous requests to support multiple concurrent users;
• Streaming output: Enables true streaming responses from agents, which is key to delivering a good user experience.

In short, you can run agent scripts casually for personal use, but enterprise production requires deploying the agent to an ASGI backend and exposing it to end users through a frontend — whether that's Vue, a mobile app, or a mini-program.

ASGI (Asynchronous Server Gateway Interface) is a server interface specification in the Python web ecosystem for handling asynchronous requests, and serves as the successor to the older WSGI standard. WSGI is based on a synchronous model where each request occupies a thread or process, making it ill-suited for large numbers of long-lived connections or streaming scenarios. ASGI, by contrast, is built on Python's asyncio event loop, allowing a single process to handle a large number of concurrent requests. For agent applications, ASGI's streaming support is especially critical: LLM inference results are generated token by token, and need to be pushed to the frontend in real time via Server-Sent Events (SSE) or WebSocket to create the "typewriter effect." FastAPI natively supports ASGI and, paired with ASGI servers like Uvicorn or Hypercorn, can deliver high-concurrency, low-latency streaming responses in production.

Integrating External Models and Existing Systems

This intelligent procurement assistant needs to connect to a rich set of external resources. On the model side, the primary inference model is DeepSeek, the fallback model is Zhipu AI, and a dedicated summarization model is also configured. This "primary model + fallback model + summarization model" multi-model orchestration reflects the enterprise-level trade-off between stability and cost.

Even more noteworthy is the integration logic with existing business systems. Many companies were already running their own CRM, ERP, and other business platforms long before LLMs became mainstream. In reality, enterprise-grade custom agents rarely start from scratch — they need to collaborate with legacy systems already in place.

This project uses an ERP as the example: the intelligent procurement assistant needs to read procurement and supplier data from the ERP. Rather than directly modifying the existing system, the project uses the MCP protocol and a gateway to access data from the existing ERP, keeping the integration non-invasive. This decoupled data access approach is a hallmark technique for deploying enterprise-grade agents.

MCP (Model Context Protocol) is an open protocol proposed by Anthropic in late 2024, designed to standardize communication between large models and external data sources or tools. Its core idea is similar to the standardization of USB interfaces: regardless of whether the external system is a database, an API, or a file system, it communicates with the agent through the same protocol, eliminating the need to build custom adapters for every integration. The MCP gateway acts as a routing hub for the protocol, forwarding the agent's data requests to the corresponding backend services while providing enterprise-grade capabilities such as access control and request auditing. This approach is especially advantageous when integrating with legacy ERP systems: the existing ERP requires no modification. You simply deploy an MCP Server on top of it to expose the necessary data interfaces, and the agent can then read purchase orders, vendor records, and other business data through the standard protocol — achieving truly non-invasive integration.

Sandboxes: Dual Protection for Security and User Isolation

When an agent executes tasks, it invokes various skills and may even download scripts from the network or have scripts auto-generated by the LLM. This introduces two engineering problems that must be addressed.

The first is skill execution security. In enterprise development, you cannot guarantee that every execution script is completely safe — a controlled execution environment is needed to isolate potential risks. The second is file and data isolation across multiple users. Since multiple users (e.g., Alice and Bob) may use the agent simultaneously through the frontend, intermediate files and analysis reports generated by each user must be kept strictly separate to prevent cross-access or privilege escalation.

The project's solution is to introduce a Sandbox service. The creator specifically runs the sandbox on a dedicated Linux server, started via the open sandbox server command. A sandbox is essentially a combination of multiple containers — a mature concept that has existed for over a decade.

The elegance of the isolation mechanism lies in assigning each user their own dedicated sandbox. When Alice uses the agent, the system allocates a private sandbox for her, and all intermediate files she generates stay within her container. Bob has his own separate sandbox. Because containers are naturally isolated from one another, the PPT file Alice generates is inaccessible to Bob, and vice versa. This approach solves both the secure execution problem and the natural isolation of multi-tenant data in one stroke.

Sandboxes are typically implemented using Linux container technology such as Docker, or lighter-weight alternatives like gVisor and Firecracker. Containers leverage Linux kernel mechanisms — namespaces and cgroups — to provide each process group with an isolated filesystem view, network stack, and resource quota, so that code running inside a container cannot perceive or access resources on the host machine or in other containers. In agent scenarios, LLMs sometimes generate and execute dynamic code (a.k.a. Code Interpreter capability). This "model-generated code" carries significant execution risk — if a malicious or buggy script is generated and run without isolation, it could corrupt production data or even compromise server security. By confining each code execution to an independent container, any abnormal behavior is strictly bounded within that container, and destroying the container afterward completely eliminates any side effects.

Image Registry and Supplier Data Scraping

Since the sandbox is composed of multiple containers, each container needs a specified image. The project configures a dedicated image registry because different users may require different container environments — certain users may need specific features or configurations, so multiple differentiated images can be prepared in advance for the scheduler to choose from.

Additionally, as a procurement assistant, the agent must work with supplier information. It accesses supplier websites via skills or web crawlers to scrape publicly available information such as pricing pages, providing data support for procurement decisions.

A Bird's-Eye View of Enterprise Agent Deployment

Although this project only covers the high-level framework, it clearly outlines the complete picture of enterprise-grade agent engineering: the Vue frontend handles interaction; the FastAPI + ASGI backend hosts the agent runtime; multi-model orchestration provides inference capability; the MCP protocol connects to the legacy ERP system; sandbox containers handle security and isolation; the image registry supports environment customization; and web crawlers supplement external data.

This is precisely where Harness Engineering delivers its value — it no longer treats the agent as an isolated AI feature, but instead incorporates it into a complete software engineering framework that addresses real production concerns: deployment, concurrency, security, isolation, and system integration. For teams looking to bring LLM capabilities into real business workflows, this architectural approach offers a highly practical engineering paradigm worth referencing.