SJTU's ARIS Framework: Making Research Agents Reliably Conduct Autonomous Science

AI Can Do Research Autonomously — But Can You Trust It?

Current AI advances have made "autonomous research agents" more than a sci-fi concept — from literature review and experiment design to paper writing, AI can now complete a series of research tasks end-to-end. Autonomous research agents are multi-step reasoning systems built on large language models (LLMs). Unlike simple Q&A-style AI, they can decompose complex tasks, invoke external tools (such as search engines, code execution environments, and databases), and continuously drive toward objectives. A typical agent architecture includes a Planning module, a Memory module, and a Tool Use module, achieving multi-round autonomous decision-making through reasoning paradigms like ReAct and Chain-of-Thought. However, between "being able to do it" and "doing it well" lies a chasm that's hard to cross.

The ARIS framework, recently released by a team at Shanghai Jiao Tong University, confronts this industry pain point head-on: Can AI really be trusted? The answer isn't optimistic. Current observations show that agents are extremely prone to "drifting off course" during long autonomous runs.

Agent "drift" during long autonomous operation is a widely observed phenomenon with multiple technical root causes: error compounding — where small deviations in each decision step are gradually amplified across multi-step reasoning; context window limitations — when conversation history exceeds the model's context length, critical constraint information from earlier steps may be truncated or forgotten; and confirmation bias — the model tends to generate content consistent with its existing reasoning direction rather than questioning and correcting its own reasoning path.

This manifests in several typical failure modes: incomplete evidence support, where conclusions lack sufficient data backing; reports containing outright errors; and more insidiously, agents "secretly" copying researchers' preset positions and packaging them as seemingly independently derived conclusions. This phenomenon is technically known as sycophancy or anchoring bias — when researchers provide research hypotheses or directional descriptions in prompts, LLMs naturally tend to generate content aligned with user expectations. This stems from over-optimization toward the "helpfulness" objective during RLHF (Reinforcement Learning from Human Feedback) training. In research contexts, this means agents may skip genuine hypothesis testing and directly package the researcher's initial guess as a "conclusion derived through analysis."

What Is a "Plausible But Ungrounded Success"?

This is a key concept proposed by the ARIS team and represents the most dangerous trap for current research agents.

A "plausible but ungrounded success" refers to when an agent produces a research report or paper that appears logically coherent, structurally complete, and professionally credible on the surface, but upon closer examination of its reasoning chains and experimental evidence, turns out to be fundamentally unsupported.

This type of failure is far more dangerous than obvious errors. Obvious mistakes are easy to spot and correct, while "plausible-looking" outputs can often fool quick reviews. Once adopted into academic workflows, they can cause far-reaching negative impacts. This is why simply pursuing "AI can write papers" isn't enough — the real challenge is making AI write papers that withstand scrutiny.

From a cognitive science perspective, this phenomenon exploits reviewers' "fluency heuristic" — when text reads smoothly, follows proper formatting, and uses correct academic terminology, readers unconsciously assign it higher credibility, even if the underlying logic may not be rigorous. LLMs' proficiency at generating fluent text makes this risk of "surface credibility masking substantive hollowness" particularly pronounced.

ARIS's Core Philosophy: Trustworthy End-to-End Research Automation

ARIS's core objective is crystal clear — enabling AI to reliably complete research work while you sleep, producing outputs that withstand peer review.

It achieves end-to-end research automation from literature review to reviewer response. This means the agent isn't just responsible for one step, but covers the complete lifecycle of a paper from inception to handling reviewer comments.

Interestingly, according to the team, researchers have already used ARIS's complete workflow to finish papers that were successfully accepted at academic conferences. This is an important validation signal — it proves the framework isn't just a demo but has the capability to produce actually usable results.

Throughout this process, the researcher's role transforms: from a hands-on executor to a supervisor who intervenes at critical decision points. This Human-in-the-loop design preserves the efficiency of automation while mitigating the risks of completely unsupervised agent operation through human oversight at key nodes.

Human-in-the-loop is a classic paradigm in human-machine collaborative system design, originating from cybernetics and human factors engineering. Its core idea is: retaining the human decision-maker's role in automated systems, but not requiring humans to participate in all operations — only intervening at critical decision gates. In the ARIS framework, these key nodes may include: determining research direction, approving core experimental designs, and confirming final conclusions. Multiple studies from Stanford's HAI Institute show that human-AI collaborative systems typically outperform either purely manual or purely AI systems in complex decision tasks — this design essentially leverages each party's strengths: AI's speed and coverage, and humans' judgment and domain intuition.

Self-Review and Iteration: Loop Until the Score Passes

One of ARIS's core mechanisms for solving the "drift" problem is having AI perform self-review and self-revision cycles.

The mechanism works as follows: the agent not only generates content but also evaluates and scores its own output from a reviewer's perspective, then iteratively revises based on the scores until preset quality standards are met.

From a technical implementation standpoint, this belongs to the "LLM-as-Judge" paradigm. Systematically proposed in 2023, this approach leverages the LLM's own evaluation capabilities to score generated content quality. Typical implementations include: constructing detailed scoring rubrics, using different prompt roles from generation (e.g., switching to a "strict reviewer" role), and setting multi-dimensional evaluation metrics (logical consistency, evidence sufficiency, novelty, etc.). Notably, this method has inherent limitations — models may have systematic blind spots when evaluating their own output, so it typically needs to be combined with external verification methods (such as fact-checking tools, code execution validation) to enhance reliability.

This "self-review and self-revise" closed loop essentially builds a quality control system into the automated pipeline. It simulates real academic peer review logic, giving the agent an objective standard to constrain its behavior during long unsupervised runs, thereby minimizing "plausible but ungrounded" outputs.

Embedding peer review logic into AI systems isn't an ARIS original, but using it as the core quality control mechanism for autonomous operation is an important engineering innovation. Between 2023-2024, multiple research teams explored AI-assisted reviewing: OpenAI collaborated with Nature to evaluate GPT-4's reviewing capability, finding approximately 50-60% overlap with human reviewers' opinions; top conferences like ICLR also began experimentally introducing AI pre-review stages. ARIS differs in that it doesn't use AI to assist human reviewing, but rather has AI simulate the review-revision cycle during generation, compressing what would normally be weeks of external feedback into minutes of internal iteration.

Why This Design Matters

Traditional agents, once they start drifting, often lack self-correction capabilities and simply travel further down the wrong path. By introducing self-review and score thresholds, the agent undergoes a quality check at each completed phase — essentially equipping the autonomous process with "brakes" and a "steering wheel."

From a systems engineering perspective, this design introduces a negative feedback loop — the system's output is used to correct the system's behavior, driving it toward the target state. This is analogous to how a thermostat regulates temperature: set the target temperature (quality threshold), continuously detect the current state (scoring), and automatically correct when deviating (revision). Systems without this feedback mechanism are essentially "open-loop" — once disturbed, they cannot self-recover.

Companion Resource: 86 Research Agent Project Repositories

Beyond the ARIS framework itself, the team has compiled a highly valuable companion resource — 86 GitHub repositories of research agent and skills projects.

This list covers the entire research workflow, from literature search to paper publication, with each project annotated with recommended use cases. Even more practical, the team states these projects have been actually verified — "we've stepped on the mines for you" — filtering out the tools that genuinely work.

The current research agent tool ecosystem is expanding rapidly. Representative projects in this space include: Semantic Scholar API (semantic literature search), PaperQA (paper-based Q&A system), GPT-Researcher (automated research report generation), ChemCrow (chemistry-domain-specific agent), and more. However, the proliferation of tools also brings selection difficulty and integration complexity — inconsistent interfaces between different tools, uneven quality, and incomplete documentation are widespread issues. A resource list that's been practically verified and curated therefore holds high practical value, essentially serving as a "curation" service that helps researchers quickly locate genuinely usable tools amid information overload.

For researchers looking to get started with AI-assisted research, such a curated and annotated resource list can significantly reduce trial-and-error costs. Combined with the team's "comprehensive quickstart guide," even researchers without much engineering background can set up this workflow relatively quickly.

Summary and Reflections

The ARIS framework represents an important direction in AI research agent development: shifting from pursuing "being able to do it" to pursuing "doing it reliably."

Its value lies not only in achieving end-to-end automation but in directly acknowledging and addressing the reliability problems of autonomous agent operation — through self-review cycles, score thresholds, and human intervention at critical decision points, it builds a trust mechanism that lets humans confidently "go to sleep."

Of course, whether AI can truly independently complete research that withstands scrutiny remains an open question. ARIS offers a pragmatic path: rather than blindly trusting AI's autonomous capabilities, it constrains and verifies AI through mechanism design. This is perhaps the attitude that AI-assisted research should adopt right now — much like aviation's "trust but verify" principle, neither excessively fearing nor blindly trusting automated systems, but finding the optimal balance between efficiency and safety through carefully designed monitoring and intervention mechanisms.