Why Do AI Agents Crash the Moment They Go Live? A Practical Guide to Harness Engineering for Runtime

Why Your AI Agent Crashes the Moment It Goes Live

Have you ever run into this situation: an AI Agent that dazzled during the demo phase crashes, spirals out of control, and gives nonsensical answers the moment it goes live? Behind this lies a core proposition worth remembering: most problems don't lie in the model, but in the system.

The capabilities of GPT-4 and Claude 3.5 are obvious to everyone. But when you connect a model to the real world—letting it operate files, execute commands, and access the network—boundary issues, security vulnerabilities, and memory gaps all surface. Demos can ignore these things; production systems cannot. This is exactly the core problem that Harness Engineering aims to solve.

This article is based on a deep dive into the official Claude Code source code, systematically mapping out the runtime engineering system for production-grade AI Agents.

What Is Harness Engineering

The concept of Harness Engineering borrows from the "Test Harness" idea in traditional software engineering—building a complete scaffold around a core module so it can run safely and predictably in a controlled environment. In the early days of software testing, a Test Harness referred to an auxiliary framework used to drive the module under test and collect its outputs, allowing developers to focus solely on the core logic itself without worrying about whether external dependencies were ready. Before CI/CD pipelines became widespread, test harnesses were already standard infrastructure for large software projects. Today, this idea has been extended to a new frontier—in the AI Agent domain, it has been expanded into comprehensive wrapping and governance of model behavior, covering multiple dimensions such as tool invocation, permission control, and state management, transforming the model from a "random black box" into a "predictable component." This shift also happens to correspond precisely to the industry's key transition point from "model experimentation" toward "systems engineering."

Harness Engineering can be understood as the runtime engineering framework for AI Agents, and it brings three paradigm upgrades:

• From model worship to systemic reflection: no longer just thinking about swapping in a stronger model, but making the system robust;
• From prompt engineering to runtime engineering: rules are written into the system, not into the prompt;
• From single conversations to production-grade closed loops: requiring persistence, observability, and automated verification.

Claude Code is the best practical example of this philosophy. Its overall architecture, from bottom to top, is divided into the entry layer, core engine layer, tool layer, service layer, UI rendering layer, and infrastructure layer, mapping completely onto the five-layer structure of Harness Engineering: the environment layer, the tool layer, the control layer, the memory layer, and the evaluation layer.

A Layer-by-Layer Breakdown of the AI Agent's Five-Layer Architecture

The Environment Layer: Giving AI a Real World to Work In

The responsibility of the environment layer is to ensure the AI no longer operates purely on paper. Claude Code builds four execution environments:

• File system environment: reading, writing, and editing files, with support for pagination, encoding detection, and image and PDF processing
• Shell terminal environment: cross-platform execution of Bash/PowerShell
• Network environment: web search and content fetching
• Code repository environment: LSP language services and Git integration

Worth mentioning is the LSP (Language Server Protocol) integrated into the code repository environment. LSP is an open protocol designed by Microsoft in 2016 for VS Code, subsequently adopted by nearly all mainstream editors including Eclipse, Vim, and Emacs, becoming the industry standard for code intelligence. Its core value lies in decoupling language intelligence features such as code completion, go-to-definition, and reference lookup from the editor, providing them uniformly through an independent language server—any tool, including AI Agents, can obtain semantic data like symbol definitions, type information, and call relationships through a unified interface, without having to implement a parser separately for each language. Claude Code's integration of LSP means the AI Agent can understand the semantic structure of code like an IDE—knowing where a variable is declared, which callers a function has—rather than treating source code as plain text. This is crucial for precise code modification and large-scale refactoring tasks, and it's one of the most fundamental engineering gaps between an ordinary ChatBot and a production-grade Coding Agent.

Take file reading as an example: it supports automatic encoding detection, offset and limit pagination reads so large files won't blow up the context, and images are automatically compressed to fit within token limits. Shell execution first performs a security analysis to determine whether a command is dangerous, then sets timeouts based on command type—30 seconds for fast commands, 5 minutes for build commands—and intelligently truncates output that's too long. It's precisely these engineering details that determine whether an Agent can run stably in a production environment.

The Tool Layer: A Unified Interface That Is Both Powerful and Controllable

Claude Code has more than 30 built-in tools, all implementing a unified interface, including Name, Inputs (parameters defined with Zod), the Call execution method, and critical security fields: CheckPermissions (permission check), IsConcurrentSafe (concurrency control), IsDestructive (destructive operation flag), and InteractBehavior (interrupt behavior definition).

Among these, using Zod to define tool input parameters is an engineering decision worth focusing on. Zod is a widely used runtime schema validation library in the TypeScript ecosystem—here it's important to understand a key piece of context: TypeScript's type system only takes effect at compile time, and the compiled JavaScript remains dynamically typed at runtime, providing no type guarantees for external inputs whatsoever. Zod rebuilds type constraints at runtime through declarative schemas, so that external inputs—whether they come from an API, a user, or an AI model—must pass explicit validation before entering the business logic. Introducing Zod into the AI Agent's tool layer means that JSON invocation instructions generated by the model must pass strict type validation before they are actually executed: whether field types match, whether required parameters are missing, whether enum values are legal. This fundamentally eliminates parameter format errors, type confusion, and similar problems caused by model "hallucinations," keeping the uncertainty of model outputs outside the execution layer.

Tools are divided into six categories by function: file operations, code search, shell execution, network access, AI collaboration, and task management. The loading strategy adopts progressive disclosure—core tools (Bash, Find, Read) load immediately, complex tools (Agent, LSP) load lazily, working together with ToolSearchTool for dynamic discovery, achieving fast startup and small context footprint.

The Control Layer: The AI Agent's Safety Guardrails

The control layer is the security line of defense for the entire runtime system. The permission decision flow is: first check AllowRules—a match means allow; then check DenyRules—a match means deny; then check AskRules—a match means ask the user; the default is also to ask. Rules can come from the project level, the user level, or be added dynamically at runtime—security rules are hardcoded into the system, rather than relying on the AI to voluntarily follow the prompt.

The control layer also includes two major mechanisms: sandboxing and interruption. The sandbox design follows the Zero Trust security principle—this concept was proposed by Forrester Research analyst John Kindervag in 2010, and after 2020, with the release of the NIST SP 800-207 standard, it became widely adopted across the enterprise security domain, with Google's BeyondCorp project being its most well-known industrial implementation. Its core logic can be summarized as "never trust, always verify": it does not default to allowing a request just because it comes from an internal system or an authenticated identity, but instead performs explicit authorization checks for every access, fundamentally rejecting the traditional assumption that "the intranet is safe." Introducing this principle into an AI Agent's sandbox design means that even tool invocations initiated by the model itself must undergo item-by-item review by permission rules. Dangerous commands and network access automatically enter the sandbox, which restricts file read/write paths, network access, CPU, and memory; violations are logged or even blocked, eliminating the risks of prompt injection attacks and model privilege escalation at the architectural level. Interruption control, meanwhile, distinguishes by tool type: cancellable tools interrupt immediately, while blocking tools wait to complete.

The Memory Layer: Externalizing State

The model's context window is limited, and the memory layer's task is to externalize state. Session persistence uses the JSONL format—a file format that stores one independent JSON object per line, widely used in big data processing, logging systems, and streaming data transmission (Elasticsearch, the consumer output of Apache Kafka, and OpenAI's training datasets all adopt this format). Compared to a monolithic JSON array, its write operations are append-only, naturally supporting streaming writes and crash recovery—even if the process terminates abnormally mid-write, the already-written lines remain fully valid, and after the system restarts it only needs to continue from the last valid record. This shares the same design philosophy as a database's WAL (Write-Ahead Log): mainstream databases like PostgreSQL and SQLite rely on WAL to achieve crash recovery and transaction atomicity. Each message is appended as one line in real time, so nothing is lost even in a crash, and it supports resuming history.

File caching adopts an LRU (Least Recently Used) eviction strategy: when the cache reaches its capacity limit, it prioritizes removing the entries that haven't been accessed for the longest time. This precisely aligns with the locality principle of code editing—developers tend to repeatedly operate on the same batch of files over a period of time. Setting a dual cap of at most 100 cached files and 25MB strikes an engineering balance point between hit rate and memory overhead, avoiding repeated reads.

Most worthy of attention is the Memory.md system: it is the entry file, forced to stay concise (at most 200 lines, 25KB), writing only the index and key decisions, pointing to topic files like Architecture.md and API-Design.md. When the AI starts up, it loads only the lightweight index and reads the detailed files when needed—this is the application of progressive disclosure to knowledge management, and the AI can automatically update it to form long-term memory.

The Evaluation Layer: Achieving Closed-Loop Feedback

The core of the evaluation layer is the Hook system, comprising four types:

• PreToolUse: parameter validation before a tool invocation
• PostToolUse: checking output quality after the invocation
• PreCommit: automatically running tests and linters before a commit
• Stop: forcing task completion when the session stops

When validation fails, the system injects a prompt to force the model to correct itself, which is equivalent to giving the AI an automatic quality inspector. The evaluation layer also has structured output validation: define a simple output schema (OK: Boolean, Reason: String) and use hooks to force the model to call it at the end of a response, making the output programmable rather than reliant on human interpretation.

Four Key Runtime Engineering Mechanisms

Sandbox Isolation

The sandbox follows three principles: zero trust, least privilege, and isolated execution. Path pattern resolution distinguishes between absolute paths, config-directory-relative paths, the user home directory, and the current working directory. Once a violation occurs, the system logs it, notifies the user, and decides whether to block based on configuration.

Sub-Agent Architecture

The core idea of sub-agents is the context firewall. The main agent delegates complex subtasks to specialized sub-agents that run in independent contexts without polluting the main context. After a sub-agent completes, it returns only the final result, with all intermediate thinking and tool invocations hidden.

Particularly worth noting is that sub-agents even support isolated execution via temporary Git Worktrees. Git Worktree is a feature introduced in Git 2.5 (2015) that allows the same repository to check out different branches or commits in multiple directories simultaneously, with each directory having its own independent working area while sharing the same .git object store. This feature was originally designed to solve the pain point of developers frequently switching contexts when working on multiple feature branches at once, and it's already a common engineering tool in Monorepo and CI/CD parallel build scenarios—modern CI systems like GitHub Actions also adopt a similar approach, using lightweight workspace clones instead of full repository copies to ensure isolation while significantly reducing storage and time overhead. Claude Code uses it for sub-agent concurrency isolation: creating a temporary worktree for each sub-agent so that multiple concurrent subtasks can independently perform file modifications in physically isolated directories, avoiding write conflicts while maintaining unified management of the repository history—this is a classic engineering practice of turning Git's native capabilities into Agent concurrency infrastructure.

Compression and Offloading

When the volume of tool output data is large, the system does not put it directly into the context but instead saves it to a file, placing only a file path reference in the context, which the AI reads when needed. When the context becomes too long, compression is triggered to summarize historical messages into a summary. This mechanism enables the AI to handle a volume of information far exceeding the context window.

Six Design Principles and Three Core Takeaways

Claude Code's source code fully embodies six design principles: minimize what the model needs to remember, write rules into the system rather than the prompt, keep tool interfaces simple, task state must be persisted, the system must be observable, and governance rules must support dynamic updates.

For developers who want to build production-grade AI Agents, this system offers three core takeaways:

1. From model worship to systemic reflection—most problems don't lie in the model, but in the system;
2. The harness is a barrier that cannot be replicated—a runtime system adapted to a specific business is harder to replace than the model itself;
3. Engineering-oriented thinking matters more than the prompt—write rules into the system, persist state, keep interfaces simple, make the system observable.

If you want to build a production-grade Agent, it's recommended to start with three steps: design a unified tool interface (including permissions, timeouts, and interruption), implement state persistence (session recovery and file caching), and embed a hook validation closed loop. These three steps don't require rewriting everything—you can evolve gradually on top of your existing system.

Remember: an Agent that works reliably is far more valuable than an Agent that is smart but constantly makes mistakes.

Key Takeaways

Key Takeaways