OpenAI Chief Research Officer Mark Chen: Within Three Years, AI Will Independently Conduct End-to-End Scientific Research

In an unconventional interview, OpenAI Chief Research Officer Mark Chen shared his deep insights on the frontiers of AI research—all while cooking Korean tofu soup. This conversation, playfully dubbed "delivering soup to researchers" (inspired by rumors of Zuckerberg poaching researchers by delivering soup), unexpectedly became an excellent window into OpenAI's research philosophy.

This article distills the most valuable insights from the conversation: from the development path of research talent, to the boundaries of reinforcement learning (RL), to OpenAI's most ambitious three-year goal—enabling models to independently conduct end-to-end scientific research.

From Trader to Research Leader: How Research Taste Is Cultivated

Mark Chen's career trajectory is quite representative—he was once a trader before becoming a core research figure at OpenAI. When asked whether there's a connection between trading and research, his answer shattered the myth that "you must have a PhD to do research."

He emphasized that OpenAI has always believed in "developing talent" over "filtering by credentials." The truly scarce ability is creatively solving problems and thinking outside the box. As for the value of a trading background, he believes its uniqueness lies in the fact that "you can't cheat"—you can't deceive the real world. It's an extremely difficult hard metric to optimize, while simultaneously demanding obsessive attention to detail and the endurance to "squeeze every last bit of return from the system." These traits naturally transfer to research.

Reproducing Papers Is the Best Path to Developing Research Taste

For those wanting to enter AI research without a background in the field, Mark offered a concrete suggestion: fully reproduce papers you admire. He recalled that when reproducing ResNet and CNN architectures in 2018, striving to precisely match the training loss and perplexity implied in the papers taught him a wealth of "hidden tricks never mentioned in the paper."

Perplexity is one of the most commonly used evaluation metrics in language models, measuring the model's uncertainty in predicting the next token—lower perplexity means more accurate text modeling. Pursuing exact perplexity matches when reproducing papers often means mastering learning rate scheduling, weight initialization, data preprocessing, and numerous other engineering details not elaborated in the paper's main text. This "tacit knowledge" is precisely what distinguishes excellent researchers from ordinary practitioners.

What truly ignited his interest was the moment AlphaGo defeated Lee Sedol—especially the famous "Move 37." In this historic match in March 2016, AlphaGo played a move in Game 2 that violated thousands of years of human Go conventions. Initially dismissed as a mistake by professional players, it later revealed strategic depth far beyond human intuition. AlphaGo combined Monte Carlo tree search with deep reinforcement learning, and its self-play generated innovations that proved AI can not only imitate human wisdom but discover paths humans never imagined. What's striking now is that Mark observes "Move 37"-style breakthroughs emerging in nearly every domain: mathematics, computer science, programming. Many people had a "sudden awakening" earlier this year, realizing that agents can already accomplish meaningful long-horizon work in their professional fields.

The Boundaries of Reinforcement Learning: Objective vs. Subjective Domains

Mark clearly delineated the current capability boundaries of reinforcement learning. RL is one of the core branches of machine learning, with a basic framework where an agent learns optimal behavior by interacting with an environment and adjusting its strategy based on reward signals. In the context of large language models, RL is primarily applied as RLHF (Reinforcement Learning from Human Feedback)—the model generates responses that are scored by a reward model, then optimized through algorithms like PPO. The core bottleneck of this paradigm lies in the quality and reliability of the reward signal.

Mark pointed out that RL performs exceptionally well in objectively judgeable domains (such as math and programming)—because answers are either right or wrong, there exists a "cold hard truth." In these domains, reward signals are naturally clear: whether code passes test cases or a mathematical proof is logically rigorous can be automatically and precisely verified, providing high-quality training signals for RL.

In subjective domains, however, RL faces inherent resistance. Take creative writing as an example—two experts might have diametrically opposed evaluations of the same piece. These "hard to score" tasks make RL difficult to apply directly. The fundamental reason is that when the reward model itself has systematic biases or inconsistencies in evaluation criteria, RL optimization can lead models to "game" the reward model rather than genuinely improve quality—this is the so-called "reward hacking" problem. While the industry is developing related techniques (such as multi-dimensional scoring, Constitutional AI, etc.), the domains where RL truly takes off remain those hard science areas where correctness is unambiguous.

AI's "High Context" Capability as Seen Through Programming Collaboration

Regarding why programming is easier for AI to tackle than "junior consultant" work, Mark's explanation is direct: programming's context is accessible (the codebase and the task itself are right there), while consultant work's context is often scattered and ambiguous. Programming task context typically exists in structured form within code repositories, documentation, test suites, and version histories that models can directly read and understand; whereas the key information in consulting work often resides in unstructured meeting conversations, interpersonal relationships, organizational politics, and other "dark information" that current models find extremely difficult to access and encode. OpenAI views programming/collaboration as a key domain for testing whether models can learn in high-context, real-world, long-horizon scenarios.

Beyond Human: How to Evaluate Superhuman Intelligence

As models begin to saturate top-tier tests like the IMO (International Mathematical Olympiad), a fundamental question emerges: How do you evaluate intelligence that surpasses the top 0.1% of human ability?

The IMO is the world's highest-level mathematics competition for secondary school students, with approximately 600 top contestants from over 100 countries participating annually. Problems span algebra, combinatorics, geometry, and number theory, typically requiring creative proof construction rather than formulaic calculation. In 2024, DeepMind's system achieved silver medal level. When benchmarks considered the "ceiling of human intellect" begin to be saturated by AI, traditional evaluation methodologies face a fundamental challenge—we cannot use human problem-solving ability as a yardstick to measure intelligence that exceeds it.

Mark's answer points to "interaction with the real world." OpenAI's early direction was pushing models toward real research—having them discover new theorems and advance hard science frontiers. Now, models solving highly difficult problems is "no longer surprising" and is even taken for granted. The next pursuit is: can models establish novel and insightful connections across different domains—for example, discovering that a theorem in topology can be applied to protein folding problems, or transferring game theory frameworks to immunology research. This kind of cross-disciplinary creative association is the capability AI currently lacks most and finds hardest to evaluate.

The Evaluation Crisis and the "Benchmark Gaming" Trap

Mark acknowledges the industry is in an "evals crisis." Classic gold-standard benchmarks like SAT are nearly all saturated, while truly high-quality benchmarks are extremely scarce. There's also the phenomenon of "bench maxing"—overfitting to specific distributions without reflecting true generalization capability. "Bench maxing" refers to research teams intentionally or unintentionally optimizing for the data distribution of a specific benchmark, causing models to score extremely high on that benchmark but plummet in performance on slightly different real-world scenarios. This is analogous to students "memorizing past exam questions" rather than truly understanding the material—exam scores are high, but the ability to solve new problems hasn't actually improved.

OpenAI's response strategy is quite instructive: separating the team that creates evaluations from the team that optimizes models. The evaluation team's goal is to build tests sufficiently difficult for models, forming an "adversarial process" (similar to the attacker-defender relationship in cryptography), aligning incentives on both sides and avoiding self-deception. They also collaborate with external institutions to create gold standards in mathematics and science.

Scaling Laws Haven't Ended: The Conviction Behind Exponential Curves

Facing the popular narrative that "pre-training is dead," Mark's stance is clear: strongly disagree.

Scaling Laws were systematically articulated by OpenAI in 2020, stating that neural network performance follows power-law relationships with model parameter count, training data volume, and compute—showing approximately linear continuous improvement on logarithmic scales. This pattern has held across nearly 10 orders of magnitude from millions to trillions of parameters, and its predictability is one of the most remarkable empirical findings in AI research history. The debate around whether it will end mainly focuses on whether physical constraints (compute, energy, data) constitute insurmountable bottlenecks.

Mark points out that this narrative has appeared repeatedly throughout LLM development history—each time someone claims "scaling can't continue because of some bottleneck," but OpenAI always manages to break through via better engineering, more refined data processing, or new research insights. Scaling Laws have held across nearly 10 orders of magnitude, and there's no reason to believe they'll stop.

Reasoning Model o1: A High-Conviction Bet

Mark uses o1 (the reasoning model) as an example of how OpenAI bets on non-consensus directions. o1 represents a fundamental innovation in LLM training paradigms—traditional models spend fixed computation per token generation ("fast thinking"), while o1 introduces the concept of "test-time compute," allowing the model to perform long chains of internal reasoning steps before answering ("slow thinking"), similar to how humans repeatedly verify their thinking when facing difficult problems. This opens a new performance improvement path independent of pre-training scaling—even without increasing model parameters, investing more computation during inference can significantly improve performance.

Before o1's creation, "pre-training + post-training" was an extremely successful paradigm, and even within OpenAI, people questioned: "If the machine already works, why forge a different path?"

It was precisely because of the firm conviction and vision of people like Jakob and Ilya toward the reasoning direction that the team truly committed. Even so, considerable "steering" was needed to get the entire company to embrace this fundamental bet. This reveals OpenAI's core Alpha—the willingness to take high-risk bets—which is also why they've maintained their frontier position. "Alpha" here borrows from financial terminology, referring to excess returns beyond market averages; in this context, it means OpenAI's source of differentiated advantage over competitors.

Three-Year Roadmap: Enabling AI to Do Research Independently

The most striking revelation from the interview was Mark's three-year goal: enabling models to conduct end-to-end independent research.

"End-to-end independent research" means an AI system can complete the entire research workflow: from literature review, hypothesis generation, experiment design, writing and running code, analyzing results, to writing papers and judging the significance of findings. This is fundamentally different from current AI-assisted research—currently models primarily serve as "tools," whereas end-to-end research requires models to possess the ability to autonomously set goals, plan long-term strategies, and adjust direction from failures, while also needing breakthroughs in long-term memory, continual learning, and multiple other technical bottlenecks.

Mark painted a picture of a rapidly approaching world—where research work will primarily shift toward "orchestration": researchers are responsible for proposing ideas, while models are powerful enough to independently handle implementation and execution. This is the "Vibe Researcher" concept—similar to "Vibe Coding" (where developers only describe intent and AI completes all code implementation), researchers need only provide directional guidance and taste judgments, while models handle all the heavy lifting of execution.

Taste and Execution: Human Researchers Remain Irreplaceable

However, Mark emphasizes that current models "don't yet possess research taste." This is precisely why human researchers still need to propose ideas—teaching models good taste is extremely difficult. "Research taste" refers to the meta-cognitive ability to judge what problems are important, what directions are promising, and what results are meaningful—this ability comes from years of accumulated domain intuition, understanding of research community dynamics, and grasp of aesthetic standards like "elegance" and "depth," which are currently extremely difficult to convey through training data or RL reward signals. But at the level of accelerating execution, the benefits are already clearly visible.

He also discussed OpenAI's unique attitude toward failure: even if a technical bet fails, the researcher's written documentation is extremely valuable, because failed ideas often "look natural," and thorough documentation helps others avoid repeating the same mistakes. For researchers with consecutive failed bets, he views them with a "trader's mentality"—as long as the expected value is positive, occasional "big hits" are sufficient to prove their worth. This reflects "power law distribution" thinking: the value distribution of research output is extremely uneven, with a few breakthrough results creating far more value than the sum of many ordinary works.

Architecture Unification: Multimodality Should Live Under One Model

Regarding whether images, audio, video, and text should be unified under a single model, Mark leans toward minimizing the number of architectures. This stance reflects the long-standing debate in AI research between "unified models" vs. "specialized models." The advantages of unified architecture include: different modalities can share representations (for example, understanding of the physical world can simultaneously help video generation and robot control), training and inference infrastructure only needs to be maintained as one set, and core algorithmic innovations (like attention mechanism improvements) can immediately benefit all modalities.

For research labs, the advantage of maintaining a single infrastructure stack is enormous—core research results can naturally transfer to any modality. The cost of maintaining and scaling multiple infrastructure stacks "should not be underestimated." The cost here includes not just engineering manpower but also the fragmentation of research attention—when teams need to maintain independent training pipelines, data pipelines, and evaluation systems for each modality, cross-modal synergistic innovation becomes extremely difficult.

Conclusion: Low-Hanging Fruit and the Final Window

When asked whether there's still abundant "low-hanging fruit," Mark's assessment is thought-provoking: there aren't many major new bets left. As models become increasingly powerful, we're approaching a world where models can autonomously generate innovations and conduct self-sustaining research. He believes "the window is small, but there are still some quite important ideas being attempted."

Regarding the view that "two or three breakthroughs are needed to reach AGI" (such as continual learning), Mark doesn't fully agree with this framing, but he's confident there are "many scoring opportunities" with a high probability of success. Continual learning refers to the ability of models to continuously learn from new experiences without forgetting existing knowledge, like humans do—current models are essentially "frozen" after training is complete. While they can temporarily adapt through in-context learning, they cannot truly accumulate long-term experience. If this problem is solved, it would mean AI systems can continuously evolve after deployment, which is a critical component on the path to autonomous research capability.

This conversation, conducted amid the steam of simmering soup, ultimately points to a clear signal: OpenAI believes that the era of AI independently conducting scientific research may be only three years away.