When AI Papers and Wikipedia Are 'Poisoned': The Gatekeepers' Battle

AI-generated content is spreading from social media into the very foundations of humanity's knowledge system. An article in IEEE Spectrum reveals a troubling reality: academic publishers and Wikipedia volunteers are trying to fight an almost unwinnable "poisoning" war with human effort alone.

The Birth of a Paper: Press 'Run Experiment' and 'Write Paper'

Benjamin Skuse, a freelance writer with a PhD in applied mathematics, ran an experiment: he used AI to generate a paper, then tried to get it past IEEE's defenses. The result was unexpectedly smooth.

According to Eliza Strickland, a contributing editor at IEEE Spectrum, Skuse only needed to "press 'run experiment' and 'write paper,' and then he did nothing and just dropped it into the entrance of the publishing engine." The barrier to generating a paper that looks plausible—"more or less science"—is astonishingly low.

This is not an isolated case. In the author's social media feed, about half of the content is filled with AI slop that sharp-eyed users have spotted—some people spread it inadvertently and feel embarrassed, while others share it deliberately to sow chaos.

Paper Mills and Academic Poisoning

The term "paper mill" originally referred to the gray-market industry that mass-produces, ghostwrites, and brokers academic papers. AI's involvement has upgraded this model from "human ghostwriting" to "model mass production": attackers can generate large numbers of manuscripts with complete structure, seemingly reasonable citations, and full data and charts within hours, hitting journal submission systems at extremely low cost. This behavior is often compared to "poisoning"—not stealing data, but injecting unreliable content into the knowledge base, contaminating search results and the citation chains of subsequent research. For publishers, the real threat is not just that a single fake paper gets accepted, but that once fake literature accumulates to a certain scale, reviews, meta-analyses, and automated knowledge graphs will treat noise as a factual foundation, and cleanup costs will rise exponentially.

Six Years After 'Hallucination': Trust but Verify

Since the GPT-3 era, the word "hallucination" has entered the AI vocabulary. Six years later, multiple chatbots have gone through several versions, and the author's attitude remains cautious: Claude or Gemini can often help, but their answers still cannot be trusted directly.

"Trust but verify, and you must verify." Applied to the integrity of the scientific record, this personal rule is the real challenge facing academic publishers.

At present, IEEE's manual review process plus research integrity checks can still keep pace with counterfeiters using free tools. But more advanced LLMs no longer make the low-level mistakes of the free versions—publishers are locked into an arms race: increasingly realistic slop on one side, and systems designed to detect it on the other.

Hallucination and the Detection Arms Race

"Hallucination" refers to content generated by large language models that appears fluent but is factually incorrect or fabricated, including fictitious citations, erroneous data, and nonexistent experimental results. Early models often made low-level grammatical or factual errors that were relatively easy for humans or statistical detectors to identify; but as model scale and alignment techniques improve, the outputs of new-generation LLMs increasingly resemble human writing in sentence structure, terminology, and argumentative structure, making detection significantly harder. Academia currently relies mainly on watermarks, perplexity analysis, metadata provenance, and manual review, but each method has room for misjudgment and evasion. This is the technical root of the "arms race" mentioned in the article: detectors rely on statistical traces left by models, and model iterations will continuously weaken those traces.

Wikipedia: Fewer People, Fewer Tools

Commercial publishers are not the only institutions holding the line. Wikipedia, the repository of global human-generated knowledge, is also resisting.

Strickland interviewed Ellia Lebleu, a founding member of WikiProject AI Cleanup. This volunteer project is dedicated to detecting and removing AI-generated material on the platform. They face the same problems as commercial publishers, but with fewer people and fewer tools.

"Wikipedia is known as a repository of human-generated information, and they want to keep it that way," Strickland said. "But who knows how long they can hold out."

WikiProject AI Cleanup and Wikipedia's Verifiability Principle

WikiProject AI Cleanup is a collaborative project formed spontaneously by Wikipedia volunteers, specifically patrolling, flagging, and cleaning up article content that is AI-generated or heavily rewritten. One of Wikipedia's core policies is verifiability: content must be traceable to reliable sources rather than relying on an editor's personal judgment. AI-generated text happens to bypass this line of defense—it may fabricate sources, patch together citations, or graft real sources onto false statements, forcing patrollers to check every item one by one. Volunteers usually identify content using clues such as edit-history patterns, abnormal language style, and dead source links, but faced with a massive volume of edits, both manpower and automated tools are severely insufficient, making Wikipedia more passive than commercial publishers in governing AI content.

When Shared Reality Begins to Crumble

If the scientific record is no longer credible and basic facts about the world are no longer accurate, our understanding of "what is real" will slide into uncertainty. This may be interesting to quantum scientists, and it is even better news for scammers profiting in an information environment without a shared reality.

But in the end, uncertainty erodes the trust foundation of the institutions that are supposed to curate the corpus of human knowledge and drive scientific progress for the benefit of humanity.

Strickland raised a sharper scenario: in a world where AI does science and AI writes papers, whether humans trust the output may not matter at all. "Isn't it crazy to think that at some point AI will be writing for other AI? We'll just be left out of the loop."

The outcome of this gatekeeping battle is still unclear. The only certainty is that the cost of verification is shifting from institutions to every reader.

Shared Reality and Knowledge Curation

"Shared reality" refers to the minimum consensus among members of a society on basic facts and public information, and it is the prerequisite for public discussion, scientific collaboration, and institutional trust. When knowledge production and curation are continuously infiltrated by low-cost, highly realistic AI content, the foundation of shared reality is weakened: people can no longer agree on "what is an established fact" and instead rely on their own information cocoons. The "curation" mentioned in the article refers precisely to institutions' function of filtering, verifying, and organizing knowledge—academic publishers, encyclopedias, and news media all play this role. Once this function fails because verification costs become too high, the trust crisis will not remain confined to individual entries or papers but will spread across the entire knowledge infrastructure.

One-line Summary

AI-generated content is disrupting academic publishing and Wikipedia, and knowledge gatekeepers are trapped in a hard-to-win verification battle.

Paragraph Summary

Based on an IEEE Spectrum report, the article describes how AI-generated content has spread from social media to knowledge infrastructure such as academic publishing and Wikipedia. Benjamin Skuse, a PhD in applied mathematics, used AI to generate a paper and got it past IEEE's initial defenses with almost zero human intervention, exposing the fragility of publishing safeguards. More advanced LLMs no longer make the low-level mistakes of free versions, and publishers are caught in an arms race between detection systems and realistic slop. Wikipedia's WikiProject AI Cleanup volunteers fight the same problem with fewer people and tools. The article warns that if the scientific record and basic facts are no longer credible, shared reality will crumble, public trust in knowledge-curation institutions will be eroded, and the cost of verification is shifting from institutions to every reader.