SciMARS: A Multi-Agent Recursive Self-Improvement Framework for Closed-Loop Scientific Discovery

SciMARS uses a shared evidence graph and decoupled component editors to make multi-agent collaboration structures recursively adaptive.
To address the lack of shared memory and the persistence of errors caused by fixed roles and processes in multi-agent scientific discovery systems, SciMARS proposes a shared evidence graph to unify observation records, decoupled component editors to separately adjust role coordination and memory strategies, and candidate probes to guide harness composition and selection, achieving recursive task-local adaptation. On Autonomous Science Bench, ScienceAgentBench, and AppWorld, SciMARS + Memory clearly outperforms baseline multi-agent teams, e.g., solving 12/18 GPT-6-Astra attempts on Autonomous Science Bench (versus 8/18). The article argues that closed-loop research needs evolvable collaboration structures, but current evaluations remain limited to specific benchmarks and a limited set of models.
Multi-agent systems are playing an increasingly important role in scientific discovery, collaboratively handling tasks such as analysis, experiment design, and validation. However, existing systems generally face two bottlenecks: agents lack shared memory, leading to duplicated work; and fixed role assignments, handoff processes, and memory strategies allow errors to persist even after the evidence has been updated. The recently proposed SciMARS framework attempts to solve these problems with a combination of a "shared evidence graph + component editors," achieving recursive task-local adaptation.

Core Design: Shared Evidence Graph and Decoupled Component Editors
The core idea of SciMARS is to decouple "evidence" from "process" in multi-agent collaboration. It introduces a shared evidence graph that lets all agents access a unified record of observations, avoiding information loss and duplicated work. On this basis, the framework uses separate component editors to adjust roles, coordination mechanisms, and memory strategies independently.
Specifically, SciMARS itself is responsible for adjusting roles and coordination methods, while the SciMARS + Memory version adds a third editor dedicated to optimizing how agents record, retrieve, and receive evidence. This decoupled design allows the system to correct errors in the collaboration process in a targeted way without disrupting existing evidence.
To guide the composition and selection of the harness in the next round, the framework also uses candidate probes for guidance. This mechanism enables the multi-agent team to dynamically adjust its own structure based on task progress, achieving recursive self-improvement.
Shared Evidence Graph is a structured knowledge base for multi-agent collaboration, typically organized as a graph (nodes represent evidence entities such as observations, hypotheses, and experimental results; edges represent supporting, contradictory, or temporal relationships). Unlike each agent's private context memory, it centrally stores and uniformly indexes the observations produced by all agents, and any member can read from and write to it. This has two direct benefits: first, it prevents the same evidence from being repeatedly collected or reasoned about by different agents, reducing compute and time overhead; second, it ensures a consistent factual basis for reasoning, reducing conflicting conclusions caused by information asymmetry. In research tasks that require long-term iteration, the evidence graph can also serve as a traceable "experimental ledger," giving subsequent process corrections a basis to work from.
Harness (execution framework/assembly) in such systems refers to a configurable execution unit composed of multiple agents together with their respective role prompts, coordination protocols, and memory interfaces. It can be understood as "the complete assembly plan for one collaborative experiment": who takes which role, how tasks are handed off among members, and how evidence is retrieved and passed in are all determined by the harness configuration. The key innovation of SciMARS is treating the harness as an object that can be edited rather than a hard-coded program, so that after each round of tasks the system can reconstruct the next round's harness composition based on performance. Candidate probes are lightweight schemes used for exploratory evaluation among several candidate configurations, helping the system determine which harness combination is more likely to succeed and thereby directing the limited trial budget toward more promising configurations.
Experimental Validation: Performance Gains Across Multiple Benchmarks
The research team evaluated scientific discovery capability through two approaches: experimental data replay and digital twins. On Autonomous Science Bench, SciMARS + Memory solved 12 of 18 GPT-6-Astra attempts and 2 of 18 GLM-5.3 attempts, whereas the baseline multi-agent team solved only 8/18 and 1/18.
On ScienceAgentBench and AppWorld, SciMARS + Memory solved 14 of 38 tasks and 11 of 20 tasks, respectively, while the baseline team scored 11/38 and 5/20. These results cover both scientific discovery and interactive task scenarios, indicating that the combination of shared memory and harness adaptation has cross-domain generalization capability.
Digital Twin in the evaluation context of this article refers to a high-fidelity virtual replica built for a real scientific experiment or engineering system, on which agents perform experiment design, operation, and validation, thereby testing scientific discovery capability without consuming real resources or bearing safety risks. Alongside it, data replay feeds historical experimental records back into the system in chronological order to examine whether agents can reproduce or advance conclusions within an existing stream of observations. The two approaches are complementary: replay focuses on the use of existing evidence and consistency of reasoning, while digital twins focus on interactive decision-making and closed-loop validation, so using both provides a more comprehensive measure of the framework's performance in scientific discovery scenarios.
Implications: Closed-Loop Research Needs "Evolvable Collaboration Structures"
The value of SciMARS lies not only in specific metric improvements but also in the direction it reveals: multi-agent systems in closed-loop scientific discovery cannot rely solely on fixed role scripts. As experimental evidence accumulates, the system needs to be able to recursively adjust its own collaboration structure—including who is responsible for what, how information flows, and how memory is retrieved.
The shared evidence graph solves the problem of "what is seen," the decoupled editors solve the problem of "how to change," and candidate probes provide guidance on "where to change." Together, the three enable multi-agent teams to maintain evidence consistency while gaining the ability to learn from errors and reconstruct their processes. For research tasks that require long-term iteration, this evolvable collaboration structure may be more practically meaningful than simply scaling up model size.
Of course, current evaluations remain concentrated on specific benchmarks and a limited set of models, and the noise, uncertainty, and interdisciplinary collaboration complexity of real research scenarios have not yet been fully examined. Nevertheless, the framework approach offered by SciMARS provides a noteworthy paradigm for deploying multi-agent systems in the field of scientific discovery.
Background Notes
Recursive task-local adaptation is key to understanding SciMARS's design philosophy. "Local" means the system does not overturn the collaboration structure as a whole but instead makes targeted modifications to the specific component that went wrong (a certain role, a certain coordination rule, or a certain memory retrieval method); "recursive" means such modifications occur repeatedly across multiple rounds of tasks—after each round, the system evaluates based on evidence and results, generates a new harness configuration, and puts it into the next round, forming a self-improvement loop. Compared with one-off manual design of agent workflows, this approach allows the system to continuously correct errors as evidence accumulates, but it also places higher demands on the reliability of evaluation signals: if the feedback itself is noisy, recursive correction may amplify rather than eliminate errors, which is a challenge the framework must face as it moves toward real scientific research scenarios.
One-line Summary
SciMARS uses a shared evidence graph and decoupled component editors to make multi-agent collaboration structures recursively adaptive.
Paragraph Summary
To address the two major bottlenecks of multi-agent scientific discovery systems—lack of shared memory and fixed roles and processes that allow errors to persist—SciMARS proposes a framework that unifies observation records with a shared evidence graph, separately adjusts role coordination and memory strategies with decoupled component editors, and uses candidate probes to guide harness composition and selection, achieving recursive task-local adaptation. Experiments were conducted on Autonomous Science Bench, ScienceAgentBench, and AppWorld, where SciMARS + Memory clearly outperformed the baseline multi-agent team on multiple metrics, for example solving 12/18 GPT-6-Astra attempts on Autonomous Science Bench (versus 8/18 for the baseline team). The article argues that closed-loop research needs evolvable collaboration structures, but current evaluations remain limited to specific benchmarks and a limited set of models.
Related articles

Why Microsoft CEO Nadella Still Backs Nvidia After Jensen Huang's Departure
Microsoft CEO Nadella publicly expresses confidence in Nvidia interim CEO Colette Kress. This article analyzes Nvidia's power transition after Jensen Huang's departure, Microsoft's compute dependency, and the trust logic of the AI supply chain.

Nadella: Critical Infrastructure Is 'Unpatchable' as AI Cybersecurity Arms Race Escalates
Microsoft CEO Nadella addresses AI cyberattack fears, saying much critical infrastructure is unpatchable, downplaying the Outlook outage, and analyzing AI's dual role in cybersecurity.

Musk Backs Jensen Huang: Firing Him Is Like Cutting Jordan
Musk says Jensen Huang is loyal to the U.S. and likens firing him to cutting Michael Jordan—highlighting politicization anxieties in the AI chip industry and defending talent value.