EvoSkill v2: How a Stateful Evolver Lets AI Agents Rewrite Their Own Skills

Improving AI agent capabilities usually relies on stronger base models or more refined prompt engineering. But a study called EvoSkill v2 proposes another path: letting agents rewrite the "skills" (i.e., instructions) they are given during operation, while the model itself and the runtime framework remain unchanged. This three-agent system improves holdout set pass rates by 7.7 and 5.4 percentage points on the SpreadsheetBench v2 and τ³-bench banking benchmarks, respectively, while achieving a lower evolution cost per unit of gain than two mainstream baselines.

Three-agent division of labor: adaptation, solving, evolution

EvoSkill v2's workflow revolves around three roles. The user first describes a benchmark task and provides a grader; a "benchmark adapter" converts it into a benchmark-specific plugin. Then a "solver" attempts the task, while an "evolver" reads the solver's training trajectories and rewrites the skill instructions each epoch.

The core of this design is decoupling: what is optimized is the skill text, not model weights or the runtime framework. This means the same evolution mechanism can be transferred to different models without retraining.

Holdout set and pass rate are key metrics when evaluating such methods. The holdout set refers to the subset of data not used during training/evolution and used only for final evaluation, to test whether skill improvements generalize rather than overfit to training trajectories. Pass rate is the proportion of samples on which the task is successfully completed. The statement "holdout set pass rate improves by 7.7 percentage points" means the absolute success-rate gain brought by the evolved skills on unseen tasks, which reflects the method's true value more than scores on the training set.

The grader is the signal source that drives skill evolution. It receives the solver's output and gives a quantifiable score or judgment, based on which the evolver determines which skill formulations are effective and which lead to failure. The reliability of the grader directly determines the direction of evolution: noisy or overly coarse scoring can cause the evolver to learn incorrect patterns, which is one reason the article later emphasizes "accepting each round's skill rather than selecting the best among noise."

Stateful evolver: one run, continuous memory

After surveying existing skill optimizers, the research team pointed out two common designs: one, like SkillOpt, uses a series of single model calls to complete optimization; the other, like SkillBoost, uses a fresh agent that starts from scratch each round. EvoSkill v2 chooses a third path—implementing the evolver as a stateful coding agent that persists throughout the entire run, equipped with bash tools to work continuously.

The direct benefit of being stateful is continuity of memory. The evolver does not need to re-understand the task context and previous failure lessons each round; instead, it can accumulate judgments about the solver's weaknesses. This is similar to a human engineer gradually narrowing down a problem within the same debugging session, rather than losing context every time they restart.

The difference between stateful and stateless agents lies in whether internal state is preserved across multiple interactions. A stateless design starts from scratch on each call and can only rely on the current input; a stateful agent maintains a continuously updated context (such as conversation history, file system, and working memory), enabling it to accumulate observations and revise hypotheses within the same session. EvoSkill v2's evolver uses bash tools to write intermediate analyses into a persistent environment, thereby preserving judgments about the solver's weaknesses across epochs. This design sacrifices some parallelism in exchange for a more coherent optimization trajectory.

Measured gains: advantages across benchmarks and feedback settings

Under fine-grained ground truth conditions, skills evolved by EvoSkill v2 raise the holdout set pass rate above the stronger of SkillOpt and SkillBoost: 7.7 percentage points higher on SpreadsheetBench v2 and 5.4 percentage points higher on τ³-bench banking.

Cost efficiency is also noteworthy. Across the two benchmarks and two feedback settings, EvoSkill v2's evolver costs less than the optimizers of both baselines for each percentage point of holdout set pass rate improvement. This shows that the stateful design is not only more effective but also more advantageous in compute overhead.

SpreadsheetBench v2 and τ³-bench banking are two complementary types of agent evaluation benchmarks. The former focuses on spreadsheet operations, requiring agents to understand table structure, execute formulas, and perform data transformations, testing precise multi-step operation capabilities; the latter belongs to the τ-bench series of tool-use benchmarks, simulating bank customer service scenarios, requiring agents to call APIs to complete user requests while complying with business rules, testing instruction following and tool use. They represent "structured data processing" and "tool use under rule constraints," respectively. Gains across both benchmarks indicate that the skill evolution mechanism is not effective for only a single task type.

Fine-grained ground truth means that scoring not only judges whether the final answer is correct but also provides detailed feedback on intermediate steps or partial results. In contrast, coarse-grained feedback may provide only a binary success/failure signal. Fine-grained signals provide the evolver with richer gradient information, enabling it to locate specific failure points; the article tests both feedback settings precisely to examine whether the method still holds when feedback is sparse.

Weak models can also self-evolve

A more illuminating result is that weaker models can also improve their own skills through this mechanism. In the τ³-bench banking experiment, when Claude Haiku 5.5 drove both the solver and the evolver, skill improvement reached as high as 21.5 percentage points.

The significance of this number is that skill evolution does not have to rely on a top-tier model as a "coach." Weaker models can also obtain significant gains in the process of reading their own trajectories and rewriting instructions. For cost-sensitive real-world deployment scenarios, this offers a feasible approach to self-improvement using the same model.

Design trade-offs: accepting each round's skill rather than choosing among noise

There are two more methodological details in EvoSkill v2 worth noting. First, the team measured the degree of trajectory fluctuation of the solver across different attempts and gave the evolver multiple attempts on each training task. Second, the system accepts the skill produced in each epoch rather than selecting among multiple candidates based on noisy scores.

The latter point especially reflects an engineering judgment: when the scoring signal is unstable, "selecting the best" based on noisy scores may introduce misleading choices, whereas continuously accepting and iterating is more robust. This is consistent with the overall philosophy of the stateful evolver—trusting cumulative improvement rather than single-point optimality.

Implications for the skill optimization paradigm

The value of EvoSkill v2 is not only in the numbers on two benchmarks. It demonstrates a direction for optimization at the skill level: treating instructions as evolvable objects and continuously refining them with a stateful agent, rather than relying on one-shot generation or round-by-round restarts. The results on cost efficiency and weak-model self-improvement provide preliminary evidence for the feasibility of this direction in real-world scenarios.

Of course, this method depends on reliable graders and benchmark adaptation, and the upper bound of skill evolution is also limited by the quality of the scoring signal. If future research can validate its performance on more open tasks with sparser feedback, it will be more convincing.

One-line Summary

EvoSkill v2 uses a three-agent system to let agents rewrite their own skill instructions during operation, significantly improving pass rates across benchmarks at lower cost.

Paragraph Summary

EvoSkill v2 proposes a skill optimization path that does not rely on stronger base models or prompt engineering: a benchmark adapter, a solver, and an evolver collaborate to rewrite only the skill instruction text while keeping the model and runtime framework unchanged. Its core innovation is implementing the evolver as a stateful coding agent that persists throughout the entire run, equipped with bash tools to continuously accumulate judgments about the solver's weaknesses, distinguishing it from the two mainstream designs of single calls and round-by-round restarts. On SpreadsheetBench v2 and τ³-bench banking, holdout set pass rates are 7.7 and 5.4 percentage points higher than the strongest baseline, respectively, and the evolution cost per unit of gain is lower. When Claude Haiku 5.5 drives both the solver and the evolver, improvement reaches up to 21.5 percentage points, showing that weak models can also self-evolve. Methodologically, the system accepts each epoch's skill rather than selecting the best based on noisy scores, reflecting trust in cumulative improvement.