Is Agent Harness Dead? The Value of Harness as Models Get Stronger

Key points: Has the Agent Harness been made obsolete by smarter models? Former Cursor engineer Jediah Katz gives a counterintuitive answer in his AI Engineer NYC talk: only the outdated layers of the Harness have been killed off, while the Harness itself is opening up a new frontier of tasks.

From Prompt to Harness: The Evolution of a Concept

Jediah Katz's career happens to sit right at the inflection point of the AI engineering paradigm. When he joined Cursor in early 2025, he was responsible for work still called "Prompt Engineering" at the time; by mid-2025, the role had evolved into "Context Engineering"—programmatically assembling the optimal context for the model; and only recently has the industry formally adopted "Agent Harness" and "Harness Engineering" to name what he and his team had been doing all along.

What is a Harness? Jediah uses a vivid metaphor: it is the model's "eyes and hands." Users connect to the Harness through the application, and the Harness connects to the model, the computer, and third-party services like MCP. He breaks it into two layers: the Runtime layer (controlling where the agent loop runs and how state is stored, ensuring work is recoverable) and the Context & Orchestration layer (deciding which model to call and what context the model sees, including prompts, tools, memory, compaction, and so on). The former is more like traditional software engineering, while the latter is the "Wild West"—and it is the part most people mean when they talk about the Harness, and it is precisely Jediah's battleground.

Context Engineering refers to the engineering practice of constructing the optimal input context for a model programmatically without modifying model weights, covering system prompts, tool descriptions, retrieved memory, conversation history compaction strategies, and more. It differs from earlier Prompt Engineering: the latter focused on hand-crafting the wording of a single prompt, whereas Context Engineering treats context as a resource that must be dynamically assembled and managed at runtime, emphasizing trade-offs and ordering within a limited context window. As agent tasks grow longer, context keeps expanding, and how to compact history, when to inject memory, and how to keep tool schemas lean all become engineering problems independent of model capability—which is the core work of the Context & Orchestration layer in the Harness.

The Evidence Chain for "Harness Is Dead"

The peak came around the New Year. Cursor announced that it had built a browser from scratch using a long-running Harness called Grind Swarm—a multi-model Harness orchestrating large numbers of agents and machines at extremely high cost, which at one point even brought down Cursor's own infrastructure.

Yet just a few months later, the community's sentiment shifted abruptly. Anthropic's Tariq talked about removing a large number of system prompts from Claude Code; YC's Gary Tan argued that Harnesses will fade and only skills matter; Jediah himself and his colleague Stefan also wrote about Cursor's process of removing many "guardrails"—such as the small model once needed specifically to apply code edits to files (because frontier models weren't good at it), and various coding-specific instructions (how to explore the codebase, how to check your work).

The common argument across these voices: smarter models are killing the Harness.

The evidence seems strong. A UC Berkeley study tested multiple Harnesses including Pi, Codex, and Claude Code with seven frontier models, and found that on newer, smarter models, the choice of Harness had no significant effect on SWE-bench Lite scores. Another study from Apple and EPFL likewise found that a specialized MLE Harness made no big difference on MLE-bench.

Jediah's response is disarmingly candid: "A fancy Harness really doesn't add coding intelligence today. That's true, and I know it's surprising."

SWE-bench Lite is a trimmed subset of SWE-bench containing hundreds of GitHub issues from real open-source repositories, requiring the system to generate patches that pass the corresponding tests. It is widely used as a benchmark for measuring a coding agent's ability to solve real software engineering problems end to end. MLE-bench, by contrast, shifts the benchmark to machine learning engineering: tasks come from Kaggle competitions and require the agent to complete the full pipeline of data exploration, feature engineering, model training, and submission. What these two benchmarks share is that they evaluate the "system as a whole" rather than "a single model output," making them naturally suited to testing the Harness's contribution—when researchers hold the model constant and only swap the Harness, the difference in scores reflects the marginal value of the Harness layer.

The Data Flywheel: How the Harness Makes Itself Obsolete

Why does this happen? Jediah introduces a core concept: the data flywheel.

The Harness makes increasingly difficult coding tasks accessible, more users can effectively attempt these tasks with agents, and thus more coding data is shared with the labs, and the models eventually learn these skills natively—after which the Harness can delete the very things it added to get the task working in the first place.

This is a self-obsolescing loop. That is precisely why people say "the Harness is dead."

But Is the Harness Really Dead? Two Reasons to Say No

Jediah's answer is no, for two reasons.

Reason 1: The Harness Is Opening a New Task Frontier

He uses Cursor's new product Grokbot as an example. It is an "autonomous colleague" that does not just coding but all kinds of knowledge work. Its Harness uses a long-lived coordinator bot to orchestrate a fleet of worker agents—like hiring a senior lawyer who deals with you directly and owns the strategy, but delegates the concrete tasks to a team of paralegals.

The interesting twist: Cursor keeps deleting prompts and guardrails in the coding context, but in Grokbot it adds these things back. The reason is that models haven't yet mastered the tasks in the new domain: making slides, booking travel, autonomous communication (keeping you informed without overwhelming you), the boundary between acting and asking (risky operations like emailing a colleague should be carried out according to your intent), and learning your working style through memory. These tasks may be familiar in coding agents, but for Grokbot's users they are common and critical—and right now it is the Harness that makes them feasible.

This completes the last piece of the data flywheel: once capabilities like coding are commoditized by models, it is precisely the Harness that unlocks new frontier tasks and starts the flywheel again. The new frontier may lie outside coding, or in the parts of software engineering work beyond coding—after all, the higher your position, the less code you write.

Another example is the "software factory" vision: multiple agents coordinating to complete the full software development lifecycle. Inside Cursor, when a bug report hits a channel, automation immediately triages it, spins up a cloud agent to attempt a fix, the agent leaves video evidence of before and after the fix, and after deployment the rollouts product spins up another agent to monitor production issues. This is like a "meta-Harness" orchestrating many agents working together.

Reason 2: Some Parts Models Cannot Absorb

First is infrastructure and runtime. Over the past year or so, agent task durations have grown from minutes to hours, days, or even longer. This cannot run on someone's laptop—the laptop will go to sleep. A truly autonomous colleague must run in the cloud.

Second is trust and security. A simple Harness with just bash and plaintext API keys really can solve coding tasks, but you probably don't want to hand all your API keys to the model—and even if you did, your security lead would have something to say. Enterprises need permissions scoped down to specific agents and specific people, and they need guardrails to avoid waking up one morning to a catastrophic operation or a $10,000 API bill.

Finally, efficiency. Although both aforementioned studies found that Harness choice has little effect on evaluation scores, both also found significant differences in cost per task. Last month Jediah focused on optimizing the efficiency of the Grokbot Harness, tuning variables such as cache hit rate, token efficiency, and model mix to significantly reduce cost and latency without sacrificing quality, with positive user feedback. As you do more and more work with agents, the most efficient Harness will only become more valuable.

Shims in software engineering originally referred to a thin layer of compatibility code that bridges interface differences without changing the underlying system. In the Harness context, shims refer to deterministic patch-like logic—for example, inserting a nudge when a model responds too slowly, forcing a retry when a particular tool call fails, or post-processing to correct output format. They exist because current models are not yet reliable enough in certain steps, and teams don't want to wait for the next model version. The problem with shims is that they become obsolete as models improve, and piling up too many makes the Harness hard to maintain, so they need to be centrally registered and continuously monitored for trigger frequency, removing any shim that no longer fires for a long time.

Next Steps: Four Trends for the Harness

The flywheel is accelerating, and the product is once again running ahead of the models. This means the Harness will get messier before it gets cleaned up, and more features will ship quickly. Teams therefore need to design the Harness in a way that makes features easy to add and remove. One manifestation is that fewer and fewer of the Harness's internal mechanisms are exposed to users—under Hyrum's Law (anything exposed to users will eventually be depended upon), reduced visibility involves trade-offs but lowers change-management burden and speeds up iteration.

Design primitives for agents. Primitives are composable, extensible building-block interfaces. With primitives, adding a new feature doesn't make the agent more confused, and removing a feature doesn't require complex refactoring. For example, the "subscription" primitive in Cursor's various agent Harnesses: an agent needs to be woken to respond to events (someone posts in Slack, CI turns red on a PR, a specific point in time), and rather than building a separate integration for each scenario, it's better to provide a unified framework and tool that lets agents subscribe to events.

Centrally manage "shims." Deterministic guardrails are essentially patches—for example, a little nudge when Grokbot isn't responding fast enough. Teams don't want to keep these hacks, but they also don't want to wait for the next model checkpoint. So they centrally track all shims, with a record every time the model uses them. There is now a bot monitoring a dashboard, and when a shim stops firing, the agent automatically files a PR saying "this shim can be cleaned up."

The Harness is moving to a higher level of abstraction. The previous generation of Harness was composed of a bunch of LLM calls strung together; new Harnesses like Grokbot and the software factory are built with agents as the core building block. Jediah also predicts that within six months, 90% of agent work will not run on laptops—this is already the established direction and a prerequisite for truly autonomous agents.

Conclusion

Back to the original question: is the Harness dead? Jediah's answer is layered: the parts that made the Harness better at coding really are dead—that's the flywheel at work; but the Harness as a way to unlock the next frontier of tasks, along with the trust, efficiency, and infrastructure it brings, will not disappear. "Yesterday's Harness may be dead, but long live tomorrow's Agent Harness."

One-line Summary

The Harness's coding-enhancement layer is being made obsolete by smarter models, but the Harness itself is opening up new task frontiers.

Paragraph Summary

Former Cursor engineer Jediah Katz responds to the "Harness is dead" argument in his AI Engineer NYC talk. He acknowledges that as models get smarter, the prompts, guardrails, and specialized instructions once added to get coding tasks working really are being removed, and studies from UC Berkeley and Apple/EPFL show that Harness choice has no significant effect on evaluation scores. He attributes this to the "data flywheel": the Harness makes tasks accessible, users generate data, models learn the skills natively, and the Harness can then delete the corresponding patches. But he argues the Harness is not dying, for two reasons: first, the Harness is unlocking new task frontiers beyond coding (such as Grokbot's slide creation and travel booking), restarting the flywheel; second, infrastructure, trust and security, and efficiency are parts models cannot absorb, and studies likewise find that the Harness significantly affects cost per task. He closes with four trends: fast feature iteration, designing primitives for agents, centrally managing and automatically cleaning up shims, and the Harness migrating to a higher abstraction built around agents as the core building block.