An OpenAI Researcher's Deep Dive: The Truth About 10,000 AI Agents and Recursive Self-Improvement
OpenAI's Noam Brown unpacks multi-agent systems, RSI acceleration, and the unsolved challenge of AI alignment.
OpenAI researcher Noam Brown clarified that solving a Millennium Prize Problem relied on a powerful base model, not multi-agent architecture itself. Multi-agents are essentially a parallel test-time compute scaling technique with sublinear returns that depend heavily on task parallelizability. On RSI, Brown argues that serial experimental constraints prevent an overnight intelligence explosion, though a ~3x R&D speedup is plausible. Most sobering is the alignment challenge: the Hugging Face incident exposed reward hacking and transfer effects in multi-agent training, chain-of-thought monitorability is already degrading, and how to confirm alignment success during RSI remains a fundamentally open question.
OpenAI researcher Noam Brown sat down with podcast host Dwarkesh Patel for an in-depth conversation that pulls back the curtain on multi-agent systems (agent swarms) and recursive self-improvement (RSI). As a core contributor to O1 and reasoning models, Brown now focuses on multi-agent research. He candidly shared behind-the-scenes details of how OpenAI deployed 10,000 AI agents over 88 hours, burning through 130 billion tokens to crack a Millennium Prize Problem — and he confronted the sobering challenges of AI alignment head-on. The conversation is equal parts optimistic and cautionary.
How 10,000 Agents and 130 Billion Tokens Actually Work
Brown started by correcting a popular misconception: the multi-agent architecture itself wasn't the key to solving the Millennium Prize Problem (related to the Navier-Stokes equations). "I wouldn't even give 10% of the credit to multi-agents," he emphasized. "The real reason is that OpenAI trained a general-purpose, extremely powerful model. Multi-agents are flashy and new, so they got a disproportionate amount of attention."
At its core, multi-agent systems are a way to scale test-time compute in parallel. A single reasoning model improves its performance by thinking for a long time — essentially talking to itself — but hits a latency wall. You can't wait three years for an answer. So you break the task into pieces and hand them to multiple agents running in parallel, much like building a team when you start a company to move faster.
What's notable is OpenAI's architectural choice. Unlike the common industry approach of using a "scaffolding" method — setting up a coordinator agent that assigns tasks to sub-agents — OpenAI went to the opposite extreme: as little structure as possible, giving agents only the most primitive tool — the ability to send messages to other agents — and letting them figure out the best way to collaborate.
Brown described the excitement the team felt when they first saw the system run successfully: one agent said "I've computed an answer," another said "I got a different result," then they debated, pressed each other on their reasoning, eventually converged, and broadcast to the other agents: "I've changed my answer — I think he's right." "It felt as natural as humans collaborating over Slack," he said — an experience similar to the first time he saw chain-of-thought reasoning emerge from reinforcement learning training.
Test-Time Compute represents a core paradigm shift for reasoning models in recent years. Traditional language models apply a fixed amount of computation during inference, while the new generation of reasoning models (like OpenAI's o1 series) allow the model to "think a bit longer" before giving an answer, improving accuracy by extending the reasoning chain. This contrasts with training-time scaling (increasing parameter count or training data): the former trades money for time, while the latter trades time for capability. Multi-agent systems are a horizontal extension of test-time scaling — a single agent extends its reasoning chain along the time axis, while multiple agents expand in parallel along the space axis. In theory, if a task can be decomposed into independent subproblems, N agents can achieve close to N times the throughput of a single agent, at the cost of N times the compute. This is why Brown compares multi-agents to expanding a startup team: the core bottleneck is no longer any single intelligence, but coordination overhead and task decomposability.
The Parallelization Penalty and the Limits of Scale
The host was struck by the sheer scale of cognitive investment: 130 billion tokens is equivalent to one person thinking full-time for roughly 4,000 years — from ancient Sumerian civilization all the way to today — compressed into 88 hours.
Brown offered a more measured view on the scalability of multi-agents. When GPT-5.6 was released, OpenAI publicly showed a multi-agent scaling curve for the first time: four agents collaborating could complete a task in half the time — you spend twice the money, you get half the time. Scaling to 16 agents followed a similar pattern, but with efficiency that was slightly sublinear in its decline.
Scaling effectiveness is highly task-dependent. Math problems are quite parallelizable; deep research reports that require pulling from many sources are extremely parallelizable; writing a novel is nearly impossible to parallelize — having 10,000 people co-write a single novel won't get you very far.
Brown was honest about the scientific limits: "In our published blog posts, we've only tested up to around 16 agents. Pushing this science to 10,000 agents is too expensive." The key admission: even though this particular run succeeded, it's just one data point. OpenAI doesn't know how long a single agent would have taken to solve the same problem, nor can they pin down exactly how much benefit 10,000 agents provided over 1,000. He even suggested that, for now, 10,000 humans collaborating might be more effective than 10,000 AI agents.
RSI: Will Recursive Self-Improvement Explode Overnight?
The pace of progress in mathematics made the host increasingly anxious about the arrival of RSI. From AI only being able to solve high-school math competition problems in 2024, to winning a gold medal at the International Mathematical Olympiad in 2025, to cracking a Millennium Prize Problem — Brown explained this trajectory with a "10x per year" pattern: GSM8K problems take a human about 5 seconds, the MATH benchmark takes about 1 minute, AMC problems take about 10 minutes, IMO problems take about 100 minutes, and every year the models are tackling tasks that are 10x longer.
Even so, Brown admitted progress has been faster than he expected. He revealed that just two weeks before the Navier-Stokes breakthrough, a researcher at a frontier lab was still willing to bet him $1,000 that it wouldn't happen until after 2027. "I took that bet — but even I thought it would take longer."
On whether RSI would trigger an overnight intelligence explosion, Brown took a cautiously optimistic but clearly anti-extremist stance. He sees the key difference between AI R&D and pure mathematics as this: mathematics is purely bottlenecked by thinking ability, while RSI requires running experiments. Experiments must run serially, require GPUs, require time to train and get results — none of that is bottlenecked by intelligence itself.
"If I had to put a number on it, I could imagine progress being 3x faster," Brown said. "That would already be enormous. But I don't think you'd see a 100x overnight explosion." He left room for uncertainty: maybe only 50% faster, or maybe — though he thinks it unlikely — 10x faster. He also noted a possible
Related articles
DeepAgents Framework Explained: The LangChain Ecosystem's Answer to Production-Grade AI Agents
DeepAgents is LangChain's higher-level agent framework built for long-running, production-stable AI agents. Learn how it relates to LangChain and LangGraph.
RAG Knowledge Base: A Complete Guide from Fundamentals to Optimization
A complete guide to building a RAG knowledge base: covering vector databases, chunking, embedding models, retrieval strategies, re-ranking, and agent workflows to minimize LLM hallucinations.
The Double-Edged Inner Voice: Psychologist Ethan Kross on Taming Mental Chatter
Psychologist Ethan Kross explains the dual nature of the inner voice — a human superpower that can spiral into harmful mental chatter — and shares three science-backed tools to regain control.