The ML Reproducibility Crisis: Three Root Causes and a Path Forward

A Scientific Foundation Being Pushed to the Margins

Reproducibility was once the gold standard of scientific research — any conclusion should be independently verifiable by others. Yet in machine learning, this principle faces unprecedented challenges. Recently, a researcher sparked a discussion on the Reddit machine learning community, bluntly asking whether "reproducibility in ML research seems to be becoming irrelevant — is it already too late?" The post struck a nerve.

This is no alarmism. When more and more research findings are "validated" only through demo videos or a handful of charts, the scientific rigor of the entire field is quietly eroding. Drawing on that discussion, this article examines the three root causes of the ML reproducibility crisis and explores possible ways forward.

Three Root Causes of the Crisis

1. Physical AI Makes Experiments Nearly Impossible to Reproduce

As research expands into "Physical AI" — robotics manipulation, embodied intelligence, and similar directions — reproducing an experiment often requires expensive hardware and even a full laboratory setup equipped with high-speed cameras.

The original post makes the point bluntly: you simply cannot confirm whether an experiment can actually be reproduced — you can only choose to trust the demo video. But demos themselves are unreliable. Researchers have a natural incentive to show only the successful clips, while the system may have crashed the moment the recording stopped. This "cherry-picking" style of presentation makes demo-based result validation extremely fragile.

When the cost of a study is so high that only a handful of institutions can afford it, the verification function of peer review has effectively broken down — no one has the means to actually rerun the experiment.

"Embodied Intelligence" refers to AI systems that learn and execute tasks by interacting with physical environments, which is fundamentally different from training models in purely digital settings. The difficulty of reproducing such research goes beyond hardware costs — it also involves the uncontrollability of physical variables: temperature, lighting, floor friction coefficients, and other factors can all affect results, and these conditions are nearly impossible to replicate precisely across different labs. In contrast, traditional deep learning experiments only require the same code and dataset running on equivalent GPU compute. The cherry-picking problem is especially pronounced in physical AI — a robotic arm grasping task might have a 40% success rate, but a researcher can simply record 10 trials, keep the 4 successful ones, and edit together a "flawless performance" demo video. Viewers have no way of knowing whether that represents the norm or the exception.

2. Big Companies' "Just Trust Us"

The second root cause comes from industry. Large AI companies frequently release tools, claiming they solve a range of problems at some accuracy or efficiency level. But unless you're an insider at one of these companies, there's no way to verify whether those numbers are real.

The problem is twofold: on one hand, these companies have strong financial incentives to inflate metrics; on the other hand, the problems they claim to have solved are often vague and subjective, lacking objective standards for verification. When a model claims a breakthrough in some "general capability," outsiders can barely define how to measure that capability, let alone reproduce it.

This creates an information asymmetry: industry controls the data, compute, and evaluation environments, while academia and the public passively receive carefully packaged conclusions.

3. Competitive Incentives Driving Deliberate Irreproducibility

The sharpest point is the third — what the original post called "the elephant in the room": researchers are actually incentivized to produce work that cannot be reproduced.

In fierce academic and commercial competition, sharing complete code and data means potentially giving competitors a leg up, or exposing methodological flaws that could be "embarrassing." This explains why so many emails sent to paper authors requesting code go unanswered.

In other words, irreproducibility is sometimes not a matter of capability — it's a rational strategic choice. When the entire incentive structure rewards "keeping something back," expecting individuals to voluntarily open up their work is almost naive.

Can History Offer Comfort?

Faced with this predicament, the original poster tried to find reassurance in the history of science: perhaps everything will be fine, because history has seen similar situations before.

For example, the Manhattan Project and the Apollo program — these projects had low outside reproducibility but high internal reproducibility: outsiders couldn't easily replicate them, but the projects themselves had rigorous internal verification processes.

However, the critical difference is this: those historical engineering endeavors were fundamentally mathematical, rigorously derived, and repeatedly verified. Much of machine learning is not. It relies heavily on empirical tuning, random initialization, massive datasets, and hard-to-articulate "engineering tricks" — it lacks the mathematical backbone that can be derived and verified from first principles.

The analogy therefore doesn't hold up: the Manhattan Project's internal reproducibility was built on reliable laws of physics, whereas the "internal reproducibility" of many ML results is itself questionable.

Machine learning and traditional engineering science differ fundamentally at the epistemological level — a point that is often underestimated. In physics or civil engineering, the validity of a design can be derived from basic equations, and the cause of failure can be traced back to a specific violated assumption. Neural networks, by contrast, tend to "work without knowing why" — a model may perform excellently on some dataset, and the researcher may have no explanation beyond "lots of data plus enough parameters." This opacity means that ML conclusions are inherently less transferable: a method that works under one lab's data distribution and hardware environment may completely fail in another, with no theoretical tools to predict such failure in advance. For this reason, "internal reproducibility" in ML deserves far more scrutiny — it cannot simply be compared to historical engineering projects backed by solid theory.

What Are the Possible Solutions?

The core question in the discussion ultimately comes down to: should reproducibility be abandoned entirely? If not, how can it be better implemented going forward?

Looking at current industry practices, despite these challenges, several paths are worth exploring.

Build Stronger Open Norms

Top venues like NeurIPS and ICML have already introduced "Reproducibility Checklists," requiring authors to submit code and describe experimental setups. Enforcement is uneven, but this at least transforms openness from a "virtue" into an "obligation," gradually shifting the incentive structure.

It's worth noting that NeurIPS began piloting a reproducibility checklist in 2019, requiring authors to describe training details, hyperparameter settings, random seeds, and statistical significance testing methods. Some top venues have also introduced "Reproducibility Tracks" dedicated to publishing replications of existing papers — whether successful or not — directly challenging the traditional bias that only positive results get published. However, the actual effectiveness of checklists is disputed: researchers who have systematically attempted to reproduce papers that passed checklist review found that success rates did not improve significantly. The fundamental reason is that much of the "tacit knowledge" influencing results — such as the fact that a certain optimizer needs a special learning rate warm-up strategy during the first few hundred steps — is difficult to fully capture in any standardized form.

Distinguish "Reproducibility" from "Verifiability"

For costly physical AI experiments, it may not be necessary to demand full independent reproduction. Instead, the field might shift toward verifiability — using standardized evaluation benchmarks, third-party audits, and open assessment protocols to allow conclusions to be checked in at least some controlled environment. This resembles the "internal reproducibility" of historical engineering projects, but requires the introduction of independent external oversight.

Resist Demo-Driven Evaluation Culture

For the research community and the public, the most important mindset shift is: reduce the weight placed on demo videos and vendor-reported numbers. A dazzling demo does not equal a reliable system, and a flashy benchmark score does not equal real-world capability. Maintaining healthy skepticism is itself a form of resistance against the reproducibility crisis.

Conclusion: Not Abandonment, But Reconstruction

Reproducibility should not be abandoned — it needs to be redefined and reimplemented within a new technological landscape. The high cost of physical AI, the information monopoly of industry, and the closed incentives driven by competition together form the triple pressure behind today's crisis.

But science's self-correcting mechanism has never stopped working — it just needs institutional design to support it. From mandatory open norms, to verifiability frameworks, to a maturing evaluation culture, each step rebuilds the scientific foundation that has been eroded. The reason this discussion about reproducibility resonated so widely is precisely because the community still cares about it. And as long as people are still asking "is it already too late?" — the verdict hasn't been written yet.