RSI Won't Happen Anytime Soon? A New Paper Uses NeurIPS Experiments to Argue No

Recursive Self-Improvement (RSI) has long been one of the most imaginative — and most unsettling — topics in AI safety and capability discussions. The scenario it describes is one where an AI system can autonomously improve itself, growing more capable with each iteration and forming an exponentially accelerating capability spiral. A new paper that sparked discussion on the r/MachineLearning subreddit throws cold water on this idea, arguing that with current AI agent capabilities, RSI is simply not imminent.

The Paper's Core Argument: Working Backward from Open-Ended Research Capability

The paper's reasoning is fairly straightforward: the prerequisite for recursive self-improvement is that an AI can independently conduct open-ended machine learning research. If a system can't autonomously design experiments, propose new methods, and validate hypotheses, it cannot truly improve itself — let alone iterate on itself round after round.

In other words, the authors treat "the ability to conduct open-ended ML research" as the necessary threshold for RSI. The key insight here is that RSI isn't about simple hyperparameter tuning or replication — it requires producing new, valuable research outcomes without a clearly defined path. If AI can't even clear this bar, then fears about RSI being "right around the corner" lack any grounding in reality.

Notably, the person who posted this paper explicitly stated they are not one of its authors — they simply found the research angle interesting. This approach of "using verifiable capability boundaries to stress-test grand narratives" is itself more worth discussing than vague optimistic or pessimistic predictions.

Experimental Design: Having AI Agents Redo NeurIPS Papers

The paper uses a clever evaluation methodology. Researchers selected a batch of papers that had been accepted to NeurIPS but not yet formally published, then asked AI agents to complete the same research work as the original papers. The resulting output was then scored and judged by the original authors of those papers.

This design has several notable advantages:

• Avoiding data contamination: Using unpublished papers significantly reduces the risk of models having "seen the answers" in their training data — a major pain point when evaluating frontier models.
• Genuine research difficulty: NeurIPS-level work represents relatively high-quality original research in machine learning, not textbook-style standard tasks.
• Authoritative evaluation criteria: Having original authors score the work reflects the actual quality and novelty of the research output far better than automated metrics.

Two representative agent systems participated in the test (described in the post using codenames referencing the Codex/GPT family and Opus family). The results: these agents were unable to complete research work at a comparable level.

Implications and Limitations of the Conclusion

Based on the finding that agents cannot handle open-ended ML research, the authors conclude that RSI will not occur at the current stage. This is an empirically grounded judgment rather than pure theoretical reasoning, which makes it more persuasive.

That said, there are aspects of this argument that warrant careful scrutiny. The first concern is timeliness: the paper tests model versions available at the time of research, and AI capabilities are iterating at a rapid pace. "Can't do it today" doesn't necessarily mean "can't do it next year." Strictly speaking, the authors' conclusion is a snapshot of capabilities at test time, not a permanent verdict.

The second concern is the equivalence of the necessary condition: does the ability to conduct open-ended research truly equate to the ability to recursively self-improve? RSI may have other implementation pathways — for instance, self-optimization within a specific narrow domain — without first requiring general scientific creativity. There's still logical room to debate whether "open-ended research" is the only valid threshold.

Third, there remains a subtle difference between "reproducing" existing research and true "open-ended exploration" in uncharted territory. A replication task with a known answer and proposing questions from scratch involve different dimensions of difficulty.

A Brief Note on Research Community Discussion Quality

Beyond the paper itself, the poster's personal reflection is also worth noting. They mentioned that they had "regretted posting research content to this subreddit about ten times before" — sometimes getting downvoted, sometimes getting upvotes but no meaningful discussion — and said this might be their last attempt.

This reflects a genuine phenomenon: in information-overloaded community environments, serious research discussions often get drowned out by emotionally-driven voting behavior. A methodologically rigorous paper that challenges mainstream narratives should ideally spark substantive conversation about questions like "how should the RSI threshold actually be defined" and "can this experiment generalize to more powerful models" — not just simple upvotes or downvotes.

Final Thoughts

The value of this paper lies not in declaring that RSI will "never happen," but in providing an actionable, verifiable evaluation framework: using real frontier research tasks to measure the capability ceiling of AI agents. For those who care about the pace of AI development, this kind of empirically-grounded perspective is more useful as a reference than emotionally-charged predictions from either camp.

Whether you lean toward "AI is about to spiral out of control" or "AI is overhyped," using "can AI actually produce a NeurIPS paper on its own" as a calibration point for your judgment is a pragmatic place to start.