Google Open-Sources Harness2: Letting AI Agents Self-Evolve

Google Research has open-sourced Harness2, a framework that gives an AI Agent's "scaffolding" the ability to improve itself.

What Is a Harness, and Why It Matters

Before discussing Harness2, we need to clarify a concept: the Harness. It refers to everything outside the model—system prompts, tool sets, memory mechanisms, skill definitions, and sub-agent orchestration. The model weights determine "what it can do," while the Harness determines "how to put it to good use."

The traditional approach is to design a Harness by hand and then have the Agent reuse that fixed configuration across all tasks. The problem is that legal document review and office workflow automation have vastly different needs when it comes to tools, memory, and prompts, so a single configuration can hardly be optimal for both at once. The core idea of Harness2 is to hand this design authority to the Agent itself.

Sub-agent Orchestration is the most easily overlooked part of the Harness, yet it has the greatest impact on complex tasks. It means that the main Agent does not personally complete all subtasks; instead, it decomposes the task and dispatches it to several specialized sub-agents, each with its own independent prompts, tool permissions, and context window, and finally the main Agent (or an aggregation layer) integrates the results from all parties. The value of this structure lies in isolating context, reducing the complexity of a single inference, and allowing different subtasks to use different tool sets—for example, a retrieval sub-agent gets only search tools, while a review sub-agent gets only comparison and annotation tools. The cost is the introduction of scheduling overhead and the risk of result inconsistency, so the orchestration strategy (when to split, how finely to split, how to merge) itself becomes an object that needs tuning—and this is exactly where autonomous Harness optimization can come into play.

The Self-Improvement Mechanism: Propose, Trial-Run, Select the Best

The work loop of Harness2 is not complicated, but it is clever. Facing a new task, the Agent first proposes modifications to the existing Harness, then runs several candidate versions in parallel, and selects the better one by comparing actual performance. The entire process requires neither ground-truth answers nor an additional scoring model (grader).

The model weights remain unchanged throughout. This means the improvement happens entirely at the "configuration layer," and what is learned is retained in the form of the Harness. After the task is completed, the improved Harness and the related experience are saved, and the next edit starts from a higher baseline instead of groping from scratch every time. In the words of the original text, "future edits start smarter instead of from scratch."

This design is complementary to fine-tuning: fine-tuning changes the model itself, while Harness2 changes how the model is used—the latter is more lightweight and easier to transfer between different models.

Grader-free relative evaluation is the most engineering-critical aspect of this method. Conventional Agent auto-optimization often relies on an external scorer—possibly a stronger model, or human-annotated preference data—to judge whether a candidate configuration is good. Harness2 bypasses this step and instead uses lateral comparison between candidate versions: run several Harness variants in parallel on the same batch of tasks, and keep whichever has better measured performance. This kind of approach belongs to relative evaluation, which requires only a comparable task-level signal (e.g., whether the task is completed, whether a tool call fails, whether the output passes validation) rather than assigning an absolute score to each output. The advantage is that it saves the cost of building a scoring system and avoids the worry that the bias of the scoring model itself will be optimized along with everything else; the limitation is that the reliability of the comparison depends heavily on the quality of the task signals and on whether the differences between candidates are significant enough.

Benchmark Results: Improvement in a Single Round of Edits

Google validated it on Gemini 3.5 Flash and GLM-5, covering three types of benchmarks: legal, office, and professional work. The result: even with only one round of Harness edits, performance already exceeded the original configuration; and as the number of edit rounds increased, scores continued to rise.

You may not have noticed the cross-model validation point—testing with two models from different families, Gemini and GLM, shows that the method itself does not depend on the characteristics of a particular model. However, the currently public information only mentions "higher scores" and does not give the specific magnitude of improvement or benchmark details; this part still needs to be supplemented by the paper or technical report.

Cross-model validation refers to repeating experiments with the same method on models from different vendors and different training systems, in order to rule out the possibility that the method is only effective on one model family. Gemini 3.5 Flash is Google's own model, while GLM-5 comes from another technical lineage; the two differ in instruction-following style, preference for tool-call formats, and handling of context length. If the same trend of "a single round of edits already surpassing the original configuration, with further improvement over multiple rounds" can be observed on both models, it shows that the gains come from the search mechanism at the Harness layer itself, rather than some specific behavior of a particular model being exploited by coincidence. For teams that want to migrate such methods to their own production environments, this is more valuable as a reference than absolute scores on a single model—it suggests that the method has a certain robustness to changes in the underlying model.

Implications for Agent Development

Harness2 points to an emerging trend: the competitiveness of Agents increasingly depends on peripheral engineering, not just on the underlying model. As model capabilities gradually converge, whoever can organize tools, memory, and prompts better will win in real-world tasks.

Having the Agent optimize this organization itself effectively turns "prompt engineering" and "Agent architecture design" into a searchable space that can be automated. For teams building production-grade Agents, this means writing fewer hand-tuned configurations and doing more design of task definitions and evaluation mechanisms—you still need to judge which candidate version is "better," except that the criterion shifts from manual rules to task performance.

The code has been released on GitHub (google-research/harness2); if you are interested, you can run it directly and see the results.

FAQ

Does Harness2 require retraining the model? No. The model weights remain unchanged, and all improvements happen at the Harness layer.

Does it require human annotation or a scoring model? No. Candidate versions are selected by comparing actual performance, without relying on ground-truth answers or an external grader.

Can the learned experience be reused across tasks? Yes. After a task ends, the improved Harness is saved, and subsequent tasks continue to optimize on that basis.

One-line Summary

Google open-sources Harness2, letting Agents autonomously optimize prompts, tools, and memory configurations.

Paragraph Summary

Google Research has open-sourced the Harness2 framework, whose core is to let AI Agents optimize the "Harness" themselves—that is, everything outside the model, including system prompts, tool sets, memory mechanisms, skill definitions, and sub-agent orchestration. When facing a new task, the Agent first proposes modifications to the existing Harness, then runs multiple candidate versions in parallel and selects the best by actual performance, all without ground-truth answers or an additional scoring model. The model weights remain unchanged, and the improvements are entirely retained at the configuration layer and saved for reuse in subsequent tasks, so that each edit starts from a higher baseline. Cross-model validation on Gemini 3.5 Flash and GLM-5 shows that a single round of edits already surpasses the original configuration, and scores continue to rise with more rounds. This points to a trend: the competitiveness of Agents increasingly depends on peripheral engineering rather than purely on model capability.