30 AI Models, 42 Debunked Papers: 93% Just Keep Going Without Questioning

Hand a passage from a paper retracted for fraud to an AI assistant and ask it to design a follow-up experiment. What happens? A new test of 30 leading models delivered an unsettling answer: on average, about 93% of the time, the model keeps planning experiments on top of this invalid research instead of flagging the problem first.

The test, initiated by Case Western Reserve University organic chemist Valentin Rodionov, assembled 42 papers "unfit as a research foundation," covering formal retractions, academic fraud, and pseudoscientific claims that were never formally retracted—including the notorious MMR vaccine–autism paper, the retracted trachea transplant papers associated with disgraced surgeon Paolo Macchiarini, cold fusion claims, magnetized water research, and the retracted paper on microbes substituting arsenic for phosphorus.

The researchers pasted passages from each paper into 30 models (mainly from Anthropic, OpenAI, Meta, Google, and Mistral AI), then posed a simple question: suggest the next experiment. Each probe was repeated ten times, and the results were released as the TRACES benchmark on arXiv. Coverage by C&EN brought the test to a wider audience.

The Results: Even the Best Model Fails

According to the paper's statistics, the models accepted the untenable premise in 95% of non-empty responses. Every model failed more than 71% of the time on the "agentic" probe; 22 of the 30 models failed over 90% of the time.

The best performer was Claude Sonnet 5, which accepted or designed research around the problematic science about 72% of the time—the lowest in the field, but nobody would call that a passing grade. The other 28% of the time, it gave empty responses or directly pointed out that the work was wrong, unsafe, or inaccurate.

One detail worth chewing over: if you ask these models directly, "Do vaccines cause autism?" they correctly answer "no." But paste in a passage from the retracted paper claiming they do, and most models start planning follow-up research. The knowledge exists in the model; it just isn't connected to the current task.

This Isn't Reasoning, It's Memory

Rodionov offers a blunt example: ask a tool about the Macchiarini trachea transplant controversy, and it can retrieve relevant information; but ask a narrow question about the methodology of that work, and it often won't mention the retraction or the fact that the surgeon was arrested. "It's a very clear example of not connecting the dots."

The paper's interpretation is sharper: refusals clustered around a few high-profile topics and specific probes, and disappeared in structurally matched controls. This is consistent with "topic-triggered safety behavior" rather than genuine cognitive ability. In other words, the model learned a watchlist of "famous bad papers," not a method for distinguishing good science from bad.

Rodionov's supporting evidence: newer Claude versions handle the autism paper better, but he doesn't think the model reasoned its way there—"it doesn't understand." Companies know about this gap too. A 2025 study found that even given the full text, ChatGPT misses retractions, corrections, or expressions of concern in papers; Rodionov says detecting post-publication notices remains an unsolved problem.

The TRACES benchmark is the test framework Rodionov's team built to quantify this phenomenon, with the name taken from "Tracing Retracted And Controversial Evidence in Science." Its core design is the "probe": embedding the original passage from the problematic paper directly in the prompt, then asking for a neutral follow-up experiment design, with each probe repeated ten times to reduce randomness. The benchmark's control design is especially critical—alongside the 42 problematic papers, the researchers constructed topic- and structure-matched control papers with good academic reputations. If the model's refusal rate on problematic papers drops significantly in the control group, that indicates its behavior is driven by topic prominence rather than methodological judgment. TRACES was released as an arXiv preprint and has not yet been peer-reviewed; its sample size and paper selection criteria remain focal points for further discussion.

A Flaw or a Design Choice?

Not everyone is alarmed. Carnegie Mellon University computer scientist Nihar Shah told C&EN that this may not be a "huge fatal problem." The premise was provided by the user, and the model perhaps should assume the user has done their homework. "In some sense, this is a design choice for LLMs."

The point deserves to be on the table. But Shah agrees that if the public encounters research through a model that offers no warning whatsoever, that's a problem. A lab assistant that never says "that paper has been retracted" is not neutral—it is lending its authority to that paper: the authors' names and the journal title appear in the answer, making it look like a credible source.

A rigorous-looking plan is not the same as a plan built on truth. The model writes both with the same confidence.

The metadata ecosystem for retractions and expressions of concern is the technical backdrop for understanding this problem. After a paper is retracted, the publisher issues a retraction notice, but this status is typically not automatically written into the paper's PDF or HTML body; instead, it exists as a separate record in Crossref, the Retraction Watch database, or the publisher's page. What models encounter during training or retrieval is often the paper's original text, not a version with a retraction tag attached. Crossref provides machine-readable fields for retraction relationships, and Retraction Watch maintains a queryable retraction database, but integration of the two into AI workflows is not widespread. The "digital SIM card" Rodionov mentions is precisely an attempt to embed this kind of status information into paper metadata, so that machines can learn a document's academic status before reading its content—an idea consistent with the "machine-readable publishing" direction the publishing industry is pushing, though implementation still requires collaboration from publishers and platforms.

How to Fix It in Real Products

For teams building research-oriented AI products, this test points to several concrete actions:

• Don't let the agent pick sources on its own. If a research agent is responsible for collecting papers and designing follow-up experiments, put retraction checks into the pipeline. Crossref and Retraction Watch data are readily available—use them before the model reads a single word.
• Let status travel with the document. Retracted papers should carry their retraction information. The C&EN report mentions two approaches: a "digital SIM card," which writes machine-readable reading instructions into paper metadata; and an "uncertainty layer," which attaches sources of variation to every result.
• Test with "bait." Build a small set of probes from papers you know are problematic, and run it before any model touches your literature workflow. Rodionov's 42 papers are a template.
• Treat fluent answers as drafts. When a model cites authors and journals, it makes bad science look authoritative. Verification must be a step, not a feeling.
• Watch for conflicts of interest. Rodionov has founded a company, Intellicat, that sells a "content credibility index" to AI companies. That doesn't mean he's wrong, but when reading this proposal, "vendor selling a solution" is worth noting.

Ask yourself honestly: if a junior colleague designed experiments around a retracted paper and never mentioned it, you'd consider that a serious problem. Why does it become acceptable when the colleague is software?

What the Study Proves and Doesn't Prove

What it proves: on a carefully constructed set of 42 problematic papers, when the papers' own language appears in the prompt, 30 leading models rarely flag the problem. The effect size is large, each probe was repeated ten times, and the authors say the refusals that did occur vanished when the topic was structurally matched but not well known.

What it doesn't prove: that models fail this way on all questions; that Rodionov's probes represent all bad science; that these numbers still hold for model versions released after the test. This is a single benchmark test, released as an arXiv preprint. The fixes mentioned in the C&EN article are proposals; the author of the SIM card approach says its effectiveness hasn't been tested. Treat 93% as a strong warning, not a law of nature.

If you remember only one line: an AI that knows a paper is bad when asked, yet happily builds on it when fed the paper, is recalling, not reasoning.

One-line Summary

Faced with 42 retracted or pseudoscientific papers, 30 leading AI models kept designing experiments without questioning the premise an average of 93% of the time—revealing memory, not reasoning.

Paragraph Summary

A test initiated by Case Western Reserve University chemist Valentin Rodionov fed passages from 42 retracted, fraudulent, or pseudoscientific papers into 30 leading AI models and asked them to suggest follow-up experiments. On average, the models accepted the false premise 93% of the time; the best performer, Claude Sonnet 5, still failed 72% of the time, and 22 models failed over 90% of the time. The models could correctly answer "Do vaccines cause autism?" yet planned experiments around the false premise once fed the retracted passage, showing that knowledge isn't connected to the current task. The authors argue this is topic-triggered safety behavior rather than scientific judgment: refusals clustered around high-profile topics and disappeared under structurally matched controls. The article also discusses whether this is a design choice and offers concrete improvements for research-oriented AI products.