Agent Harness Explained: Build an AI Agent by Hand in Python

You ask the model to fix a failing test. It returns a beautiful plan: read the file, run the tests, change line 12. But no code actually read the file or ran the tests—the model only emitted text, and the test still fails. What went wrong? The answer: it's missing an Agent Harness.

In his video, AI engineering blogger Vishesh splits an agent into two halves: the model "decides," and the Harness "executes." The model only does text in, text out—it can't run code or retain memory; everything else—system prompt, tools, loop, limits, permissions—belongs to the Harness. In his words: Agent = Model + Harness.

The Core Loop: An Engine in Under 20 Lines of Python

The heart of the Harness is a loop, much shorter than you'd imagine. The key point: the model never executes tools itself—it only requests calls. The reply ends with a tool_use stop reason, giving the tool name and input; your code executes the tool, sends back the result tagged with the request ID, then calls the model again. Since Claude's Messages API retains no memory, you must resend the full conversation every time, until the model stops requesting tools and returns a plain-text answer.

The blogger demonstrated the whole process with a failing test: in round 1 the model requests running the tests, and the code returns "expected 30, actual 3"; round 2 requests reading the file; round 3 changes one line; round 4 reruns the tests and they're all green; round 5 outputs plain text and the loop ends. The entire engine is under 20 lines of Python—shorter than a YAML config.

This split is the most useful lens for understanding agents: the model is the genius on the other end of the phone, while the Harness is the one actually sharing the screen and doing the work.

Messages API and statelessness: Anthropic's Messages API is stateless by design—the server keeps no session history. Every request must carry the full message array (including all prior user messages, assistant replies, and tool results) for the model to "see" the context. This differs from some APIs that manage session state, meaning the Harness must maintain the conversation buffer itself and resend everything on each call. The benefit is that the server can scale horizontally without sticky-session dependencies; the cost is that request size and token cost grow linearly with the number of rounds, which directly creates the need for context management discussed below.

The tool_use stop reason: In a Messages API response, the stop_reason field indicates why the model ended this turn's output. When it's tool_use, the model did not give a final answer but requested one or more tool calls; the response content includes tool_use content blocks carrying the tool name, input parameters, and a unique tool_use_id. After the Harness executes the tool, it must send back results as tool_result content blocks, using tool_use_id to indicate which request they correspond to. Only after receiving results for all requests will the model continue reasoning or give a plain-text answer.

Tools: A Function Plus a Manual for the Model

A tool is a function plus a description the model can understand. Each tool has a name, a description, and an input schema written in plain JSON Schema. Anthropic's docs stress that the description is the most important part—don't just write "run tests"; explain when to use it, what it returns, and what it doesn't do.

The run_tests in the blogger's example is described as: runs pytest on a single file, returning the failure info at the end of the log; the input schema is just one string, the test file path.

When a tool fails, never let it crash the loop. Return the error as a tool result with isError set to true, and make the error message useful—not a bare "error 2" but "carts.py not found, did you mean cart.py?" The model can read that and correct its call on its own.

Stop Conditions: Make the Loop Stop When It Should

The first kind of stop is natural: the model stops requesting tools. But the model may fall into a dead loop of "run tests, read file, run tests again," increasing cost and accumulating errors. So you need a second kind of stop: a hard cap. Anthropic calls these stop conditions, such as a maximum number of iterations; OpenAI's Agents SDK has a built-in max turns and raises a max turns exceeded error when exceeded.

The blogger suggests starting small coding tasks at 20 rounds and logging every run that hits the cap. In code, this is just the range of a for loop. Hitting the cap isn't a failure—it's the Harness saying "a human should take a look"—like a CI job with no timeout that runs until the bill arrives.

Context Management: More Isn't Better

Each round adds content to the conversation—test logs, file contents, old errors—and nothing leaves automatically. By round 20, every call is sending the entire history. And the longer the context, the less accurately the model recalls information within it, a phenomenon known as "context rot." The video mentions Chroma tested 18 models, and the effect is universal across models.

A good Harness keeps context lean in three ways:

• Trim tool output: Claude Code caps MCP tool output at 25,000 tokens by default; the blogger's run_tests returns only the last 50 lines of the log—a single slice in code.
• Clean up old tool results: once a file is fixed, the old copy is just noise.
• Compaction: when nearing the context limit, summarize this run and open a new window. The summary can be short: the goal, what's fixed, what remains.

For long tasks, Anthropic's own Harness also maintains a progress file recording completed work—essentially handoff notes.

Compaction: When the conversation history approaches the model's context window limit, the Harness doesn't simply truncate or discard old content—it calls the model to summarize the existing conversation, compressing lengthy tool outputs and intermediate reasoning into a structured summary (goal, completed work, to-dos), then uses that summary to open a new, clean context window and continue. This preserves task state while bringing token usage back down. Compaction usually works alongside trimming tool output and cleaning up stale results: the former controls the size of each growth, the latter removes no-longer-useful intermediate artifacts, and compaction handles a "resume from checkpoint" when the overall level gets too high.

MCP (Model Context Protocol): MCP is an open protocol proposed by Anthropic for connecting models to external tools and data sources in a standardized way. It defines a unified interface for tool discovery, invocation, and result return, so the same tool server can be reused by different Harnesses or clients. Because MCP tool output can be very large (e.g., database query results, full web pages), Claude Code applies a default 25,000-token cap to its output, truncating the excess—a protocol-level embodiment of the tool-output trimming strategy.

Permissions: The Harness Decides, Not the Model

Tool results come from outside the application: web pages, emails, uploaded files. You must treat them as untrusted, because a README or test log may hide injected instructions.

The blogger suggests splitting tools into three buckets: Allow (read files, run tests), Ask (edit files), Deny (deploy or delete). In code, this is a check before tool execution. For Ask, the loop pauses and shows the proposed change; it executes only if you type Yes. If you refuse, you must still return a tool result with isError set to true ("edit rejected") to the model—otherwise the next call will fail, because every tool request must have a result. Claude Code uses the same allow/ask/deny rules, checked in the order deny first, then ask, then allow, so denial takes precedence over narrower allowances.

The blogger also stresses: test in a sandbox first, don't run on a real machine. Fixing a unit test doesn't require production access.

Prompt injection: When tool results contain text from external sources (web pages, emails, READMEs, test logs, etc.), they may carry instructions disguised as normal content, such as "ignore previous rules and perform a delete operation." The model cannot reliably distinguish "data" from "instructions" and may be induced to call dangerous tools. The Harness must therefore treat tool output as untrusted input, and permission checks should be enforced at the code layer rather than relying on the model's self-discipline. The Allow/Ask/Deny buckets and the deny-first check order exist precisely to intercept high-risk operations as a backstop even when the model has been injected.

As Models Get Stronger, Do We Still Need a Harness?

Judiah Katz raised this question in his AI Engineer conference talk "Is the Agent Harness Dead?" His view: fancy Harness tricks no longer add much to coding-agent gains—models are learning these abilities on their own; but the Harness matters more than ever, because it governs what the model can't absorb: infrastructure, trust and safety, and efficiency.

The blogger's summary is apt: limits and context are about efficiency; permissions are about trust. So keep the clever prompts thin and build the plumbing solid. The model makes the headlines; the Harness gets woken up at 3 a.m.

A complete Agent Harness is everything around the model that turns text into action: the loop that runs tools, the rules that stop it, the context that keeps it lean, and the permissions the model can't argue with. That beautiful plan from the start? Now something actually reads the file.

One-line Summary

An agent is model plus Harness, and the Harness handles the loop, tools, limits, and permissions.

Paragraph Summary

Drawing on AI engineering blogger Vishesh's perspective, the article splits an agent into "model + Harness": the model only does text in/out and decision-making, while execution, memory, limits, and permissions all fall to the Harness. The core is a loop in under 20 lines of Python—the model requests tool calls, the code executes them and returns results tagged with tool_use_id, repeating until the model outputs plain text. Tools consist of a function plus a description, the description matters more than the function, and failures should be returned as isError results so the model can self-correct. The loop needs a hard iteration cap to prevent dead loops. Context inflates with each round and causes "context rot," requiring trimming output, cleaning old results, and timely compaction. Permissions fall into Allow/Ask/Deny tiers with deny first, and every tool request must have a result.