AdaRSI: On-Demand Recursive Self-Improvement Helps Embodied Agents Break Through Long-Horizon Task Bottlenecks

Pre-trained vision-language-action models (VLAs) often fail midway when facing unseen long-horizon tasks. Such failures are not merely execution errors—they expose a more fundamental question: when a system cannot complete a task, should it adjust how it invokes the executor, or change the executor's inherent capabilities?

A recent paper titled "AdaRSI: On-Demand Recursive Self-Improvement for Embodied Agents" directly addresses this question, proposing an on-demand recursive self-improvement framework.

The Dilemma of Recursive Self-Improvement

Recursive self-improvement (RSI) is considered an effective path to enhancing agent capabilities. However, in embodied scenarios, existing methods face a structural contradiction.

One class of methods chooses to freeze the executor and only optimize the "harness" around it—such as invocation logic and task decomposition strategies. This approach cannot teach the agent a new action it fundamentally lacks (a missing motion), leaving it helpless when hitting the executor's capability boundary.

Another class of methods directly fine-tunes the executor on persistently failing trajectories, yet never asks: is this failure because the executor was invoked incorrectly, or because it simply cannot do it? This "train first, ask later" approach easily misattributes problems that should be blamed on scheduling strategy as weight updates, wasting compute with limited effectiveness.

AdaRSI's core contribution is precisely treating "when to change weights" as an explicit decision problem.

What is a Vision-Language-Action Model (VLA)

VLA models are a class of embodied intelligence foundation models that unify visual perception, natural language instructions, and action generation into a single neural network. Representative examples include RT-2, OpenVLA, and GR00T, which are typically pre-trained jointly on internet-scale image-text data and large-scale robot manipulation data to achieve cross-task and cross-embodiment generalization. Users only need to provide natural language instructions and current visual observations, and the model directly outputs end-effector action sequences. The core value of VLAs lies in replacing task-specific policy design with a unified pre-training paradigm, but they also inherit the common shortcomings of pre-trained models: when facing long-horizon composite tasks outside the training distribution, they easily lose coherence midway, and because action outputs are highly coupled with weights, it's difficult to quickly diagnose whether an error is a cognition-layer problem or a capability-layer problem. This is precisely the pain point that self-improvement frameworks like AdaRSI aim to address.

How AdaRSI Works: A Three-Step Loop

AdaRSI's pipeline can be broken down into three stages, forming a closed loop.

Step one: first let the coding agent improve its own skills and harness. Before touching the executor, the system first tries to solve the problem by improving invocation methods and task decomposition. This step is the cheapest and safest.

Step two: attribute persistent failures. For sub-steps that still fail, AdaRSI retries from saved states to determine whether the problem lies in the invocation method or the execution capability. This "retry-attribution" mechanism is AdaRSI's key differentiator: it doesn't blindly flag failures but locates their true source.

Step three: fine-tune only the genuinely unreliable sub-steps. More importantly, the demonstration data used for fine-tuning is collected by the system itself and reuses experience accumulated from previous tasks.

The ordering of these three steps embodies a pragmatic principle: don't touch weights when strategy can solve it; only precisely update when weights must be touched.

The Division Between Executor and Harness

In AdaRSI's context, the executor refers to the neural network body that carries action generation capability, typically the weights of a VLA model; the harness is the scheduling and orchestration layer wrapping around the executor, including task decomposition, sub-goal generation, tool invocation order, prompt construction, and failure retry logic. The key difference between the two lies in intervention cost: modifying the harness only requires adjusting code or prompts, with no gradient updates needed, taking effect immediately; fine-tuning the executor requires collecting demonstration data, backpropagating, and updating parameters—costly and potentially introducing catastrophic forgetting. Splitting agent capability into these two layers allows "when to change weights" to be formalized as an attribution problem—when a failure falls within the harness's coverage, change the code; only when a failure proves that the executor lacks a certain motor skill is it worth touching the weights. This division provides actionable engineering guidelines for resource-constrained embodied systems.

Experimental Data: A Leap from 4.6% to 13.8%

The paper validated on all 50 target tasks in RoboCasa365, using GR00T N1.5 as the executor, without any human demonstrations throughout.

The results are quite impressive. AdaRSI improved VLA performance across all three suites, raising success rate on unseen composite tasks from 4.6% to 13.8%—nearly a threefold relative improvement.

More noteworthy is the efficiency comparison. Compared to the approach of "fine-tuning every sub-step," AdaRSI achieved slightly higher success rates while consuming only 28% of the GPU time and 22% of the self-collected demonstration data. At the other extreme—fine-tuning without harness optimization—although training volume was larger, the gains fell far short of AdaRSI.

This comparison demonstrates two things: blind fine-tuning is not only expensive but inefficient; there is a clear complementary relationship between harness optimization and precise fine-tuning, rather than being interchangeable options.

Implications for Embodied AI Development

AdaRSI's value lies not just in a specific framework, but in the problem awareness it raises: self-improvement systems need a mechanism for judging "when to change weights."

For developers, this means that when designing embodied agents, "strategy-layer optimization" and "capability-layer fine-tuning" can be treated as two independent intervention levers, dynamically selected based on attribution results. This on-demand approach is closer to the resource-optimal solution than either "fine-tune everything" or "only adjust the harness."

Of course, AdaRSI currently still relies on retrying from saved states, which holds in environments where states are reproducible, but in highly dynamic or irreversible real-world scenarios, attribution costs may rise significantly. This is also a question for future research to answer.

Overall, AdaRSI provides a more pragmatic path for embodied agent self-improvement: attribute first, then act; avoid touching weights when possible, and when necessary, touch only what's essential.

One-line Summary

AdaRSI turns "when to fine-tune executor weights" into an explicit attribution decision, achieving nearly 3x success rate improvement with significantly reduced compute.

Paragraph Summary

Pre-trained vision-language-action models (VLAs) often fail midway on long-horizon unseen tasks, and existing recursive self-improvement methods face a dilemma: either freeze the executor and only adjust the harness, failing to acquire missing motions; or blindly fine-tune on failing trajectories, wasting compute. AdaRSI proposes an on-demand recursive self-improvement framework that, through a three-step loop—first optimizing the harness, then retrying and attributing persistent failures, and finally fine-tuning only genuinely unreliable sub-steps—makes "when to change weights" an explicit decision problem. On 50 target tasks in RoboCasa365, AdaRSI raised success rate on unseen composite tasks from 4.6% to 13.8%, consuming only 28% of GPU time and 22% of self-collected demonstration data, demonstrating that harness optimization and precise fine-tuning are complementary.