Why Do Coding Agents Underperform? The Problem May Lie in the Harness

The Plight of Coding Agents: Lynch's Challenge and a Team's Practice

Well-known developer Lynch recently raised a sharp question: Why are coding agents so dumb? This question sparked extensive discussion in the developer community. After running multiple ticket-to-PR agents, one team reached a thought-provoking conclusion—the problem is mainly not with the model itself, but with the harness we build around the agent.

This view challenges the mainstream narrative. When coding agents underperform, people habitually blame the underlying large language model for lacking capability. But real deployment experience shows that even powerful models often fall short in actual development workflows. This suggests there may be systemic flaws in the "glue layer" between the model and the development environment.

What Is a Coding Agent's "Harness"?

A harness is the entire support system built around an AI model, including prompt templates, tool invocation interfaces, context management, error recovery mechanisms, and feedback loops. It determines how the model perceives tasks, performs actions, and learns from results.

A poorly designed harness can make the smartest model look foolish. For example, if the agent cannot obtain the full codebase context, or if tool calls fail frequently without a retry strategy, then no matter how powerful the model is, the final output will suffer greatly. This is the technical root behind the phenomenon Lynch questioned.

Context Window is the upper limit on the total number of tokens a large language model can process in a single inference, covering the input prompt and generated content. In coding agent scenarios, it directly determines how much code, ticket description, test output, and historical operation record the agent can "see." Too small a window misses critical dependencies, causing changes to break other modules; too large a window introduces many irrelevant files, diluting attention and driving up cost and latency. Therefore, one of the harness's core tasks is context budget allocation: deciding which information must always be present, which can be retrieved on demand, and which should be compressed into summaries or symbolic indexes. Common engineering approaches include embedding-based semantic retrieval, AST-level code slicing, dependency graph pruning, and rolling summaries. Understanding the hard constraints of the context window is necessary to see why "stuffing the entire repository into the model" is neither feasible nor economical.

From Ticket to PR: Harness Flaws Exposed in Practice

The team ran ticket-to-PR agents in real projects. These agents need to understand ticket descriptions, locate relevant code, write changes, and submit PRs. The process exposed several typical harness problems:

• Crude context window management: Agents are often stuffed with too many irrelevant files, drowning out key information; or the context is too short, missing necessary dependencies.
• Lack of robustness in tool calls: After basic operations such as file read/write and command execution fail, agents lack effective recovery paths and easily fall into infinite loops.
• Sparse feedback signals: Agents struggle to extract actionable correction information from test failures or compilation errors, leading to repeated mistakes.

These problems are not model capability flaws, but rather failures in harness design to provide a sufficiently friendly working environment for the model.

Idempotency means that performing the same operation once and multiple times produces the same result. For coding agents, if actions such as file writes, patch application, branch creation, and PR submission lack idempotency guarantees, retries will lead to duplicate insertions, conflicting overwrites, or redundant submissions. In practice, agents often enter a chaotic "retry—partial success—retry again" state due to network jitter or tool timeouts, ultimately producing dirty patches. A robust harness designs each tool call as a safely replayable atomic operation, or introduces explicit state checkpoints and transaction rollback: record a baseline snapshot before execution, compare differences after failure, and then decide whether to roll back or continue. Idempotent tool primitives combined with state checkpoints are the engineering foundation that keeps agents recoverable in uncertain environments.

Why Do We Always Love to Blame the Model?

Attributing the problem to the model is a cognitive shortcut. Model capability is easy to quantify, while harness quality is hard to measure, so it is easily overlooked. In addition, model vendors' marketing tends to emphasize breakthroughs in the model itself, downplaying the importance of supporting frameworks.

This attribution bias leads teams to frequently switch models when optimizing agents, with little effect. What truly needs investment is harness engineering: how to design more precise context retrieval, more robust tool abstractions, and denser learning signals.

Directions for Building a Better Harness

Based on the team's practical experience, improving the harness can start from several levels. For context management, use dynamic retrieval instead of full injection, letting the agent obtain code snippets on demand. The tool layer needs to provide idempotent, retryable operation primitives with built-in state checkpoints. For the feedback mechanism, structure test output and lint results before passing them back to the model so it can make targeted corrections.

These improvements do not require a more powerful model, but more meticulous engineering design. When the harness is mature enough, even a moderately capable model can perform excellently on coding tasks.

Structured Feedback means converting the output of tools such as tests, compilation, lint, and type checking from free text into machine-parseable signals, such as JSON records with file locations, line numbers, error codes, and severity levels. Raw logs are often verbose and noisy, making it hard for the model to locate actionable correction points. Once structured, the harness can return only diagnostic items relevant to the current change, sorted by severity and accompanied by fix suggestions, forming high-density learning signals. Furthermore, feedback can be associated with specific diffs, letting the model know exactly "which line change caused which failure," enabling targeted correction rather than blind rewriting. This layer of design directly determines whether the agent can quickly converge from errors.

Implications for AI Programming Tool Development

Lynch's challenge and the team's practice together point to one conclusion: the effectiveness of a coding agent is the product of the model and the harness. Ignoring the harness and simply pursuing model upgrades is like putting a stronger engine in a race car while ignoring the transmission and tires.

For developers building AI programming tools, this means more resources need to be invested in refining the agent framework. Tool invocation protocols, context compression strategies, and error recovery logic—these seemingly dull engineering details—are the key to whether coding agents can truly be deployed.

In the future, as model capabilities gradually converge, harness quality will become the core competitive advantage distinguishing excellent coding agents from mediocre ones. Teams that solve the framework problem first will see truly usable automated programming assistants sooner.

One-line Summary

Coding agents underperform mainly because of the external harness rather than model capability; effectiveness equals the product of model and harness.

Paragraph Summary

The article centers on developer Lynch's question, "Why are coding agents so dumb?" Combined with one team's practice running ticket-to-PR agents, it points out that the problem lies mainly not in the model itself but in the external harness built around it. The harness covers prompt templates, tool calls, context management, error recovery, and feedback loops, and poor design makes even strong models perform badly. Three types of flaws emerged in practice: crude context window management, lack of robustness in tool calls, and sparse feedback signals. People habitually blame the model because model capability is easy to quantify while harness quality is hard to measure. Improvement directions include dynamic context retrieval, idempotent retryable tool primitives, and structured feedback signals. The core conclusion is that coding agent effectiveness equals the product of the model and the harness, and in the future harness quality will become the core competitive advantage.