Self-Evolving Harness
A technical paradigm that enables a language model agent, with frozen model parameters, to simultaneously act as both solver and optimizer within a single harness codebase, iteratively improving its own execution framework through recursive self-improvement.
Timeline (last 90 days)
The Self-Evolving Harness framework proposed in the paper has the same frozen model act as both the solver and the proposer, editing and running its own surrounding code.
Each evolution batch in the framework samples tasks from benchmarks across five different domains to avoid overfitting to a narrow task distribution.
The harness obtained at the end of the first stage improved the average score on in-distribution benchmarks by 4.48 points and on out-of-distribution benchmarks by 12.64 points.
The first-stage performance surpassed Codex on in-distribution benchmarks and matched Codex on out-of-distribution benchmarks.
During the evolution process, three mechanisms spontaneously emerged—output truncation, history compression, and independent review—none of which were manually pre-designed.
The paper frames the entire evolution process as two stages—multi-task pre-training and continual training—drawing on deep learning training paradigms.
All Facts (6)
The Self-Evolving Harness framework proposed in the paper has the same frozen model act as both the solver and the proposer, editing and running its own surrounding code.
50%UnverifiedEach evolution batch in the framework samples tasks from benchmarks across five different domains to avoid overfitting to a narrow task distribution.
50%UnverifiedThe harness obtained at the end of the first stage improved the average score on in-distribution benchmarks by 4.48 points and on out-of-distribution benchmarks by 12.64 points.
50%UnverifiedThe first-stage performance surpassed Codex on in-distribution benchmarks and matched Codex on out-of-distribution benchmarks.
50%UnverifiedDuring the evolution process, three mechanisms spontaneously emerged—output truncation, history compression, and independent review—none of which were manually pre-designed.
50%UnverifiedThe paper frames the entire evolution process as two stages—multi-task pre-training and continual training—drawing on deep learning training paradigms.
50%