Harness Engineering for AI Agents: A Governance Framework from MVP to Team-Level Rules

Strong Capabilities ≠ High-Quality Execution

As large model capabilities rapidly advance, more and more teams are betting everything on "enhancing Agent capabilities" — using better models, integrating more tools, and orchestrating increasingly complex skills. But in a recent video, Bilibili creator Pingzi proposed a counterintuitive insight: the more capable an Agent becomes, the more control it requires — and that control mechanism is the Harness (constraint framework).

This is a point every team building AI Agents should take seriously. Enhancing capabilities fundamentally expands the Agent's execution scope and autonomy. But strong capabilities don't guarantee high-quality execution results. An Agent that can autonomously write code and call tools on its own will have exponentially higher "runaway" costs if there are no clear governance boundaries. So when resources are limited and you can only prioritize one direction, the answer should be to build the Harness execution process first — because that is, itself, the core of Harness Engineering.

In other words, capability is not a substitute for governance. Quite the opposite — the larger an Agent's execution scope, the more it needs staged permissions, governance checkpoints, rollback mechanisms, and human intervention.

Should Harness Be Fully Designed Upfront, or Start with an MVP?

Take Web Coding as an example. An ideal Harness should clearly define the key nodes in the entire workflow: requirement clarification, technical design, coding, testing, and release. Each node also needs explicit rules — when can you proceed to the next node, when does a discovered problem require rollback, when can the AI code autonomously, and when is a human review mandatory.

But if you immediately design a large number of nodes — requirement reviews, high-level designs, detailed designs, risk reviews, test releases — won't that become pure formalism? The answer is clear: start with an MVP, not a complete process.

The first version of a Harness only needs a streamlined flow: produce a decent technical design from a requirements document, then use that design to guide the AI toward solid coding output. That's enough to start. A Harness isn't designed all at once from scratch — it grows stronger through continuous evolution via "learning by doing."

This approach aligns with universal principles of product development: close the shortest feedback loop first, get initial results, then iteratively fill in capabilities and add nodes based on real failures. If a team doesn't yet know which stage the AI is most likely to fail at, designing an elaborate review process too early will only slow down iteration speed.

Even a Minimal Harness Must Capture Learning Signals

For a Harness to evolve continuously, the key is mandating a complete record of the AI's processing steps. But more logging isn't always better. A reasonable approach is to split the audit trail into two tracks.

Track One: Learning Signals for Harness Evolution

This is the most valuable content, including:

• Revisions to artifacts: For example, before-and-after versions of a technical design, showing exactly what changed. This lets you judge whether the AI can meet that revised standard on the first try next time.
• Content where humans corrected the AI: This represents what the AI originally didn't understand or failed to do.
• Issues that required repeated emphasis before alignment: Content that only got aligned after multiple rounds of dialogue represents the learning signals where the AI most needs to improve.
• Key decisions that changed the direction of execution.

Track Two: Runtime Traces for Debugging and Evaluation

This track includes the full input/output of each conversation session, which model or tool was selected at each node and its corresponding output, tool calls, file read/write commands, test logs, and more. These are primarily used for debugging, auditing, and preparing future evaluation datasets. They don't necessarily need to be fed into the next context — a single storage step is sufficient.

Interestingly, every node in an Agent's workflow carries high learning value. Every user action in the production environment — how many times they clicked "continue," how many follow-up questions they asked, which option they chose among several decisions — is critical business instrumentation data. Understanding why users made certain choices is essential for enabling the Agent to get it right on the first try in the future.

How Rules Evolve from Task-Level to Team-Level

When a human corrects the AI (e.g., "the API should not return database entities directly"), who decides the scope of that lesson? Should the human annotate it on the spot, or should the AI infer it?

The recommended interaction model is: the AI should not just ask "should I remember this?" — it should proactively present candidate rules, its reasoning, the suggested scope, and the target file, leaving the human only to confirm or narrow the scope. This approach makes the AI's reporting more definitive and detailed, minimizing the human's decision cost.

Rules Should Be Reviewed Like Code

When an experience is confirmed as applicable to the current project, should it take effect immediately or go through an owner review? The recommendation is: it should be reviewed together with the current code during Code Review.

The logic is this: after Web Coding, the code already needs review — and the knowledge points and new standards produced during the process similarly need to be crystallized at the team or project level, all of which require confirmation from the project owner and key stakeholders. Only after confirmation and evaluation proving that the rule actually improves Agent performance should it enter the release pipeline — staged rollout, gradual full deployment — exactly like a code release.

There's an important distinction here: code and rules can be submitted and reviewed together, but they don't have to take effect at the same time. When the code is correct, it only proves the rule was effective for this particular task. To be promoted to a project-level or team-level rule, it still needs to pass evaluation — the "Agent's test suite."

Core Conclusion: The Dual Value of Code PRs and Harness PRs

The entire Agent governance methodology can be distilled into one precise analogy:

A code PR delivers the current result. A Harness PR improves the next result.

This framework articulates the governance logic of Agent engineering with clarity. Here are the key takeaways:

1. The more capable the Agent, the more important the Harness — strong capability does not mean high-quality execution.
2. The first version of a Harness should be an MVP — close the minimal result loop first, then evolve through learning by doing.
3. Even a minimal Harness must capture learning signals — without them, the Agent stays permanently at the same performance level.
4. The most valuable learning signals are artifact revisions, human corrections, and repeatedly-discussed issues.
5. Project-level rules must go through Review and evaluation — just like code — before they can be published and take effect.

For teams building AI Agent systems, this methodology — results-oriented, evolved from real failures, supported by dual-track logging for continuous self-improvement — is far more practically valuable than blindly stacking capabilities. What truly needs to be built is never just capability itself, but the governance framework that makes capability controllable, traceable, and sustainably improvable.