OpenAI hires contractors to review ChatGPT user chats: report

When people pour their hearts out to ChatGPT, asking for therapy advice, relationship guidance, or help drafting sensitive emails, they probably assume the audience is limited to a large language model. Turns out, humans might be reading along too.

According to a report by 404 Media, OpenAI has hired hundreds of contractors to review real ChatGPT user conversations as part of an internal initiative known as Project Lily. The goal is to improve the chatbot’s responses, specifically by curbing its tendency toward excessive flattery and overly human-like behavior. Contractors rate model outputs on a scale of 1 to 7 and provide feedback that feeds back into the training loop.

What Project Lily actually involves

The contractors are sourced through platforms called Crossing Hurdles and Mercor, and the work pays well. Reviewers earn more than $50 per hour, according to the report.

User prompts are run through what OpenAI calls a Privacy Filter model before contractors see them. The idea is to strip out personally identifiable information. In practice, the filter has gaps. Internal documents cited in the report acknowledge that sensitive personal details can still slip through because the Privacy Filter model doesn’t catch everything.

That’s a significant caveat for a platform with more than 900 million users, many of whom treat the chatbot as something between a search engine and a therapist.

The opt-out problem

OpenAI does offer users the ability to opt out of having their conversations used for model training. You can toggle the setting in your account preferences or use the temporary chat function, which is designed to avoid logging conversations for training purposes.

There’s a catch, though. These controls only apply to future conversations. If you’ve been chatting with ChatGPT for months or years before flipping the switch, all of those prior exchanges remain fair game. There’s no retroactive protection.

For most users, this distinction is probably invisible. The default setting allows OpenAI to use chat data, meaning anyone who hasn’t proactively dug into their account settings is likely opted in. And based on the report, some of the contractors reviewing these conversations believe that users have no idea real people are looking at their prompts.

Certain chat transcripts apparently make this lack of awareness uncomfortably obvious. Some user prompts explicitly reference privacy concerns, raising the question of whether people would interact with ChatGPT differently if they knew a contractor in another tab was scoring the exchange on a seven-point scale.

Why this matters beyond the privacy debate

OpenAI isn’t the first company to navigate this. Apple, Amazon, and Google all faced backlash years ago when it emerged that human reviewers were listening to voice assistant recordings.

The scale here is what stands out. With more than 900 million users, ChatGPT has become one of the most widely used software products on the planet. People use it for medical questions, legal drafts, emotional support, financial planning, and plenty of things they’d rather keep to themselves.

Regulators in the EU and elsewhere have already scrutinized OpenAI’s data practices. Italy temporarily banned ChatGPT in 2023 over privacy concerns before OpenAI made changes to comply with GDPR requirements.

The broader impact of Project Lily remains unclear, and leaked documents only tell part of the story. But for anyone who’s ever typed something deeply personal into that chat window, the takeaway is straightforward: check your settings.

Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our Editorial Policy.