Project Lily Exposed: Who Is Reading Your ChatGPT Conversations
ChatGPT conversations aren't fully private — human reviewers reading user chats is standard practice in AI model training.
Sparked by "Project Lily" trending on Hacker News, this article exposes a widely overlooked reality: human reviewers reading user conversations is standard practice for mainstream LLM products, driven by the need for model alignment and content filtering. The real risk isn't the review itself, but a three-layered transparency gap — users often input highly sensitive content, service providers' de-identification promises have technical limits, and most users are never clearly informed or given easy opt-out options. The article advises users to adjust privacy settings and avoid sharing identifiable information, while calling on the AI industry to adopt more transparent data-use disclosure practices.
When AI Conversations Aren't Private
A report known as "Project Lily" has sparked discussion on Hacker News — its title pointing directly at a reality many users overlook: your conversations with ChatGPT may not be a secret shared only between you and a machine. Behind this project name lies a long-standing truth that rarely gets openly discussed: human reviewers read portions of users' conversations with AI.
To be upfront: the original source material this article draws from is extremely limited — just a title, 13 upvotes, and zero comments, with no detailed disclosure of Project Lily's specifics, how it operates, or who is involved. What follows is therefore less a factual reconstruction of the project itself, and more an analysis of the industry context and privacy issues it raises.
Human Review: The Open Secret of AI Training
For any mainstream large language model product, human involvement in data review is essentially standard industry practice. There's a clear technical rationale: models need reinforcement learning from human feedback (RLHF) to align outputs, filter harmful content, and correct errors. Completing this work requires human access to real user conversation samples.
In other words, the notion of a "private conversation with AI" is, in many cases, a misconception. Most AI services' privacy policies contain relatively vague language stating that some conversations may be used for training or manual review to improve the service. The problem is that the vast majority of users have never carefully read these terms — and have no idea what "some" actually means in terms of what percentage of conversations are reviewed, by whom, or under what conditions.
RLHF (Reinforcement Learning from Human Feedback) is a core component of alignment for today's mainstream large language models. The process typically involves three steps: first, fine-tuning the base model with supervised learning; then having human annotators rank or score multiple model outputs to train a "Reward Model"; and finally using a reinforcement learning algorithm (such as PPO) to guide the language model's output tendencies based on that reward model. It is in the second step that real user conversations — or deliberately constructed test conversations — must be read and evaluated by human annotators. OpenAI, Anthropic, Google, and other major players all rely on annotation teams of varying sizes to complete this work, with some tasks outsourced to third-party data service companies. This means the parties "reading conversations" aren't limited to employees within the product company itself — they may extend to external partners, further increasing the complexity of data flows and the difficulty of privacy control.
What Users Should Actually Be Concerned About
What's truly worth paying attention to isn't the act of "someone reading" conversations per se, but rather several key transparency issues.
The Sensitivity of the Data
When users converse with tools like ChatGPT, they often input far more sensitive content than anything they'd share on social media — health consultations, legal questions, financial details, even emotional confessions. Once this content enters a human review process, its sensitivity far exceeds that of ordinary web browsing history.
How Reliable Is Anonymization?
Service providers typically claim that data is de-identified before review. But research has long demonstrated that simply removing direct identifiers like names is insufficient to fully prevent re-identification, especially when conversations contain a wealth of unique personal details.
"De-identification" and "anonymization" carry distinct meanings in academic and legal contexts. De-identification merely removes direct identifiers (such as names and email addresses), while true anonymization requires that individuals cannot be re-identified even when combined with external datasets. In the early 2000s, researcher Latanya Sweeney demonstrated that just three data points — zip code, gender, and date of birth — were enough to uniquely identify approximately 87% of the U.S. population. The 2006 re-identification of Netflix's anonymized dataset became a landmark cautionary tale in privacy research. Conversational text carries a high re-identification risk due to the wealth of contextual detail it contains (specific experiences, locations, descriptions of relationships) — often higher than structured data. This makes the "de-identification protection" claimed by service providers considerably limited in practice.
The Right to Know and the Right to Opt Out
More fundamentally, the question is whether users have clear, informed awareness of what's happening and accessible opt-out mechanisms. Some products offer the option to "disable chat history for training," but these settings are often buried deep in menus and tend to be enabled by default.
How to Protect Your Conversation Privacy
Even without full control over how service providers behave, users can take some practical steps:
- Review and adjust the privacy settings of AI products, and disable the "use for model training" option if one is available
- Avoid entering directly identifiable information in conversations, such as your real name, national ID number, or bank card details
- For highly sensitive consultations, prioritize solutions that offer enterprise-grade privacy commitments or local deployment
- Regularly clear your conversation history to reduce the amount of data that persists long-term
Closing Thoughts: Transparency Is the Real Issue
The reason the title "Project Lily" strikes a nerve is that it transforms an abstract privacy clause into a vivid image: someone is reading your conversations. For the AI industry, human review is unlikely to disappear any time soon — it remains a necessary component of model quality and safety. What genuinely needs to improve is making this process more transparent: clearly informing users how their data is used, who reviews it, and how it is protected, while providing simple and accessible choices.
(Note: This article is based on limited publicly available information. Specific operational details about Project Lily have yet to be disclosed by authoritative sources. Readers should refer to official and credible media coverage for follow-up reporting.)
Related articles
Xi Jinping Proposes Open Source AI Cooperation Zone Among BRICS Nations
Xi Jinping proposed an open source AI cooperation zone at the BRICS summit. Analyzing the strategic intent, open source rationale, and global AI governance implications.
Swift-Qwen3.8-27B: 58% Fewer Thinking Tokens, Nearly 2x Faster Inference
UkisAI open-sources Swift-Qwen3.8-27B, cutting thinking tokens by 58% and boosting inference speed 1.95x via overthinking token penalties and on-policy distillation — with under 1% accuracy loss.
Netflix Partners with Sega: Crazy Taxi Movie and New Sonic Animated Series on the Way
Netflix announces three Sega game adaptations: a Crazy Taxi movie, a new Sonic animated series with edge, and a live-action film based on RGG Studio's Stranger Than Heaven.