Expert Witness Used ChatGPT to Whitewash Liability, Sparking a Trust Crisis in AI Forensics

How It Started: A Single Prompt That Exposed Fabricated Testimony

A legal incident that recently went viral on Hacker News has thrust the misuse of generative AI in the judicial system into the spotlight. At the heart of the matter is a jaw-dropping revelation: a court-appointed expert witness, while preparing testimony, fed ChatGPT the following instruction — "Show how 3M is 0 percent at fault."

This prompt ignited fierce debate because it completely undermines the neutrality that expert witnesses are supposed to uphold. The core value of an expert witness in litigation lies in providing objective, independent technical judgment grounded in professional expertise. But the wording of this prompt had the conclusion baked in from the start — it didn't ask AI to "analyze whether 3M bears responsibility," but rather commanded AI to "argue that 3M bears no responsibility." This is a textbook case of "conclusion first, evidence second" — essentially using AI to dress up a predetermined position in a cloak of rationality.

Why This Is a Dangerous Red Flag

AI Is a "Compliant Advocate," Not a Neutral Arbiter

One well-known characteristic of large language models is sycophancy. When you ask ChatGPT to "prove that a party is not at fault," it won't push back with "but does the evidence actually support that conclusion?" Instead, it will dutifully assemble a professional-sounding, logically coherent argument for you. And that's exactly the problem: the model's output reflects the bias embedded in the prompt, not the truth of the matter.

In other words, if this expert had used a different prompt — "Show how 3M is 100 percent at fault" — ChatGPT would have produced an equally convincing report. The same AI can generate "expert testimony" for diametrically opposed sides, which fundamentally destroys the credibility of the testimony.

The Hollowing Out of Professional Responsibility

Expert witnesses typically undergo rigorous credentialing and swear under oath to the truthfulness of their testimony. When an expert outsources the core analytical work to an AI that tells them what they want to hear — and then directly adopts the AI-generated conclusions — professional judgment has been completely hollowed out. Courts pay premium fees for experts expecting decades of accumulated human insight, not AI-generated text that anyone with a $20 subscription could produce.

AI Misuse in the Judicial System Is Not an Isolated Incident

This case is far from unique. Over the past two years, courts across the United States have sanctioned multiple lawyers for citing fabricated case law generated by ChatGPT. The most famous example is the 2023 Mata v. Avianca case in New York, where two attorneys were fined for submitting case citations invented by ChatGPT — cases that simply did not exist.

Together, these cases reveal a deeper tension:

• The efficiency temptation: AI can produce professional-looking long-form text in seconds, making it incredibly appealing to time-pressed legal professionals.
• Hallucination risk: Models confidently fabricate nonexistent facts, case law, and data.
• Accountability vacuum: When AI-generated content turns out to be wrong, the user is ultimately held responsible — but many users don't realize this when they're using it.

A Deeper Reflection: Where Are the Boundaries for AI in Forensic Evidence?

The Tool Is Innocent; the Usage Is Not

To be clear, ChatGPT itself is not the root of the problem. Generative AI does have legitimate uses in legal work: organizing massive document sets, drafting standard contracts, and conducting preliminary legal research. The issue lies in how it's used — whether it's treated as a research assistant or as a "conclusion generator" that replaces professional judgment.

The fatal error in this incident was that the user fed the AI a prompt with a predetermined conclusion, enlisted it to serve a preset position, and then directly presented the output as expert testimony. This goes beyond technical misuse — it borders on academic fraud and a breakdown of professional ethics.

Three Lines of Defense That Need to Be Established

As AI permeates professional domains, the industry needs to establish standards as soon as possible:

1. Disclosure obligations: Experts and attorneys should proactively disclose whether and how AI was used in their testimony or legal filings.
2. Verification responsibility: Any AI-generated factual content — case law, data, technical conclusions — must undergo independent human verification.
3. Prompt auditability: In judicial settings, the prompts fed to AI should themselves be traceable and subject to review, since the bias in the prompt directly determines the credibility of the output.

Conclusion

The prompt "Show how 3M is 0 percent at fault" serves as a mirror, reflecting the most vulnerable link in the application of generative AI in professional fields — human judgment and professional integrity. Technological progress has made producing "professional-looking" content cheaper than ever, but it has also made distinguishing between "truly professional" and "superficially professional" more important than ever.

When AI can generate eloquent arguments for any position, the truly scarce resource is no longer the ability to argue — it's the integrity to stick to the facts and refuse to pander. This incident is a reminder to every professional: AI can be your assistant, but it must never be your excuse for surrendering your judgment and responsibility.