ChatGPT users are routinely sharing personally identifiable information (PII), sensitive emotional disclosures, and confidential material with the AI platform, analysis from SafetyDetectives has revealed.
The cybersecurity reviewer’s deep dive into 1000s of ChatGPT conversations, leaked in August 2025, confirms what many already suspected: many internet users aren’t fully aware of how the AI model handles and distributes their data, and they have a startling level of trust in a still-emerging technology.
A UX Flaw That Turned into a Data Leak
The leak originally stemmed from a now-removed “Make Chat Discoverable” feature that allowed search engines to index conversations and make them accessible to anyone online. While the option did warn that chats would appear in web searches, SafetyDetectives suggests many users failed to grasp the full implications.
Users are Using ChatGPT as Therapists and Lawyers
The analysis, covering over 43 million words across the exposed conversations, found that people are treating ChatGPT as everything from a therapist to a legal consultant.
Nearly 60% of flagged conversations fell under “professional consultations,” with users asking the chatbot for advice on sensitive matters, including:
- Legal disputes
- Workplace conflicts
- Family planning
In many cases, users disclosed suicidal ideation, addiction struggles, and emotional trauma. Others uploaded full resumes, complete with names, addresses, phone numbers, and employment history. This is all information cybercriminals could use to blackmail victims or for fraud and identity theft.
Some Users Participate in Marathon Sessions
The dataset also revealed unusual and concerning patterns in user behavior. While most chats were short, coming in at under 500 words, some were startlingly lengthy. Just 100 chats made up more than 43 million words analyzed, and the longest single conversation ran to 116,024 words, roughly the length of a full novel.
Calls for Stronger Warnings and Redactions
SafetyDetectives argues that the incident highlights the need for stronger protections and clearer dislosures. Their report recommends:
- Prominent, unambiguous warnings against sharing personal or confidential data
- Automatic redaction of PII when conversations are shared
- Simplified and safer defaults for privacy settings, so users do not inadvertently expose sensitive material
Without stronger safeguards, the researchers caution, oversharing with AI assistants will continue to present a growing cybersecurity and public safety issue.
Josh is a Content writer at Bora. He graduated with a degree in Journalism in 2021 and has a background in cybersecurity PR. He's written on a wide range of topics, from AI to Zero Trust, and is particularly interested in the impacts of cybersecurity on the wider economy.
The opinions expressed in this post belong to the individual contributors and do not necessarily reflect the views of Information Security Buzz.