Google Open-Sources Harness2: Recursively Rewriting the Agent Framework at Test Time

Improving large model capabilities usually depends on weight updates, but Google's research team offers another path: leave the model parameters untouched and only modify the Agent's "shell." They have open-sourced Harness2, an Apache-2.0 licensed framework whose core idea is to recursively rewrite the Agent harness at test time. According to official disclosures, a single refinement step alone improved GLM-5 by 11.5 points on WorkBuddy^Ad-Bench.

What Harness2 Actually Changes

The Agent harness is the engineering structure wrapped around the base model, responsible for prompt organization, tool invocation, context management, retry strategies, and more. The traditional approach is for developers to manually design the harness and then fix it in place. Harness2 turns this process itself into an object that can be optimized: during the inference stage, the framework analyzes the current harness's performance, generates improvement plans, and recursively applies these rewrites.

This brings a key distinction—the model weights remain completely untouched. All gains come from changes to the harness structure. For closed-source models that cannot be fine-tuned and can only be accessed via API, the practical value of this approach is obvious.

Typical composition of an Agent harness: A complete harness usually includes the organization of the system prompt, the registration and invocation protocol for tools (tool/function), context window trimming and compression strategies, memory management for multi-turn interactions, failure retry and fallback logic, and the parsing and validation of model outputs. These components are coupled with one another, and any change in one place can alter the model's actual behavior. Therefore, the harness is not a simple "prompt template" but an engineered runtime contract. The reason Harness2 can achieve substantial gains by rewriting the harness is precisely that there is considerable design space in the above components that has not been manually tuned.

What an 11.5-Point Improvement Means

WorkBuddy-Bench is a benchmark that measures Agent performance on real office tasks. GLM-5 achieved an 11.5-point improvement on this benchmark through a single refinement step, and this number needs to be understood in context: it is equivalent to the gains from a medium-scale fine-tuning run, but the cost is only a few structural rewrites at test time.

It should be noted that this result comes from a single official source and currently lacks third-party reproduction. Whether recursive rewriting can work stably across different models and different task distributions still needs community verification.

On WorkBuddy-Bench and the meaning of "points": Agent benchmarks typically consist of a set of office tasks requiring multi-step tool calls (such as retrieval, spreadsheet processing, document generation, cross-application operations, etc.), and scoring is usually based on task success rate or weighted sub-goal completion. Therefore, the specific meaning of "11.5 points" depends on the benchmark's scoring scale: if it is a percentage-based success rate, then it corresponds to an absolute improvement of 11.5 percentage points, which is a significant magnitude; if it is some kind of weighted score, then it needs to be interpreted together with the baseline score. When reading such disclosures, the baseline value, number of tasks, and scoring script version are all necessary context for judging the value of the gains, and currently these details are still incomplete in the public information.

The Engineering Implications of Recursive Rewriting

"Recursive" is the key word in the Harness2 name and also what distinguishes it from ordinary prompt optimization tools. The framework does not make a one-time prompt adjustment; instead, it takes the rewritten harness result as input for the next round of rewriting, forming an iteration. This requires the framework to have the ability to evaluate its own behavior; otherwise, the recursion will quickly diverge or fall into a local optimum.

The choice of the Apache-2.0 license means there are no obstacles to commercial use, which is a positive signal for teams hoping to integrate the framework into production Agent pipelines. Open sourcing also lowers the barrier to reproduction, allowing the community to test the boundary conditions of recursive rewriting as soon as possible.

An extension of test-time compute: Traditionally, test-time compute refers to investing more sampling, search, or verification during the inference stage (such as best-of-n, chain-of-thought expansion, tree search). Harness2 extends this idea from "searching over answers" to "searching over harness structures"—compute is spent on rewriting and evaluating the shell rather than directly generating more candidate answers. This also means its cost structure differs from sampling-based methods: the cost mainly comes from repeatedly evaluating harness variants, and evaluation itself requires running complete tasks, so the total cost may grow approximately linearly or even faster with recursion depth, which is a key variable when assessing its cost-effectiveness.

Several Issues Worth Watching

Test-time rewriting introduces additional inference overhead and uncertainty. Each recursion requires evaluating the current harness's performance, and this portion of the computational cost is not quantified in the disclosure. Another issue is stability: whether the recursively rewritten harness will produce behavioral drift across tasks requires longer-term evaluation data.

For teams building Agent systems, the idea offered by Harness2 is more worth learning from than the specific implementation—treating the harness as an optimizable variable rather than a fixed scaffold may be a low-cost path to unlocking the capabilities of existing models.

One-line Summary

Google open-sources Harness2, which recursively rewrites the Agent shell rather than model weights at test time, improving GLM-5 by 11.5 points in a single step.

Paragraph Summary

Google's research team has open-sourced Harness2, an Apache-2.0 licensed framework whose core idea is not to update model weights but to recursively rewrite the Agent harness (prompt organization, tool invocation, context management, and other engineering structures) wrapped around the base model during the inference stage. According to official disclosures, a single refinement step alone improved GLM-5 by 11.5 points on the office-task benchmark WorkBuddy-Bench, a magnitude close to a medium-scale fine-tuning run but at the cost of only test-time structural rewriting. This path is especially valuable for closed-source models that can only be accessed via API. The article also notes that this result comes from a single official source and lacks third-party reproduction, and that the additional inference overhead and behavioral drift risks brought by recursive rewriting have not yet been quantified, but its idea of "treating the harness as an optimizable variable" is worth learning from for Agent teams.