Lessons from the ChatGPT Confession Files

The rapid expansion of generative AI tools in India’s IT and business services sector has brought both innovation and profound new risks. This was sharply revealed by the ChatGPT Confession Files, an investigation by Digital Digging, a substack with expertise in online research methods using the Web and Artificial Intelligence, which on July 31^[1] exposed how AI platforms’ “share” features that could inadvertently turn confidential business, legal, and personal conversations into searchable public records. In a fresh update, recent reports highlight how user conversations were indexed by Google through a “discoverable” toggle—an option that OpenAI has now abolished after widespread criticism and scrutiny.

These developments underscore the urgent need for India’s organisations, as global custodians of sensitive data, to overhaul policies, technology, and awareness for AI-enabled workflows in line with the Digital Personal Data Protection (DPDP) Act, 2023.

Digital Digging’s investigation demonstrated shocking gaps in user awareness and platform design: hundreds of real ChatGPT conversations – often containing business strategies, proprietary code, personal information, or compliance-sensitive data – were found indexed on open web searches. Many professionals and knowledge workers were caught off-guard, wrongly believing their shared chats were accessible only to colleagues or protected by privacy.

The issue gained renewed urgency after additional investigation revealed that thousands of ChatGPT conversations – including sensitive subjects like mental health, legal questions, and professional advice – were appearing in Google search results. Users were often unaware that checking a “Make this chat discoverable” box would broadcast their conversations to the world. Even deleted chats could linger on the web under certain legal constraints.

OpenAI has since eliminated the “discoverable” sharing function, calling it a “short-lived experiment.” Yet the episode exposes how quickly a design oversight or lapse in user understanding can lead to severe data exposure – an especially serious issue for India, where global technology clients rely on ironclad privacy and stewardship.

India’s prowess in IT services and global capability centres (GCCs) depends on maintaining client trust for handling intellectual property, proprietary information, and regulated data. The ChatGPT Confession Files and its recent updates, demonstrate that a single “share” click can erode decades of trust, trigger regulatory censure, and risk severe legal and financial fallout, including for employees acting in good faith.

With growing remote and hybrid workflows, AI tools are now routine for problem-solving, code review, documentation, and even sensitive brainstorming. This amplifies the opportunity for accidental leaks – not just with ChatGPT, but with other AI providers as well. While some platforms (like Claude or Gemini) default to stricter privacy, their policies are not uniform, and user conversations may still be used for model training unless specifically opted out.

The Digital Personal Data Protection Act, 2023, affirms that organisations (“Data Fiduciaries”) are responsible for protecting personal and sensitive data – regardless of whether leaks originate through internal error or third-party tool usage. Legal penalties for breaches are now significant, and “it was an employee’s mistake” is not a viable defence.

There are, therefore, immediate steps that can be taken at the policy, corporate and personal level to protect privacy.

For governance and policy, there must be

Explicit, organisation-wide prohibitions on feeding sensitive, proprietary, or regulated data into general-purpose AI platforms lacking robust privacy guarantees.
Mandated use of internal or enterprise-grade AI solutions for all confidential data.

Technologically, it is necessary to

Enforce technical controls to block the creation of shareable public AI links or detect such links in real time.
Monitor all use of generative AI tools and restrict access based on data classification.

To ensure safety, training and Awareness is key:

Governments, companies and academic institutions must provide mandatory and regular data privacy education for all employees and staff, incorporating real examples from the ChatGPT Confession Files and recent public disclosures.
Simulate incident scenarios so employees understand the risks associated with digital sharing features.

Most importantly, every government and company must

Build rapid response protocols for accidental exposure, including fast notification, takedown requests, and DPDP Act-compliant reporting.

The fallout from the ChatGPT Confession Files—and the subsequent abolishment of “discoverable” shared chats after search engine indexing^{^[2]}—demonstrate a hard lesson: even cutting-edge technology is only as safe as its most misunderstood feature. For India’s global technology sector, the stakes are uniquely high. Trust, legal compliance, and professional stewardship require a robust blend of policy, awareness, and vigilance—not only to prevent embarrassing headlines, but to uphold India’s place as a secure leader in the global digital economy.

The call to action is clear: review every workflow involving AI, strengthen controls, and make data protection as routine as digital innovation itself.

Brijesh Singh is Adjunct Distinguished Fellow, Cybersecurity Studies, and a senior IPS officer.

This article was exclusively written for Gateway House: Indian Council on Global Relations. You can read more exclusive content here.

For permission to republish, please contact outreach@gatewayhouse.in.

Support our work here.

References:

[1] Henk van Ess, “The ChatGPT Confession Files,” Digital Digging (with Henk van Ess), July 31, 2025. https://www.digitaldigging.org/p/the-chatgpt-confession-files

[2] “ChatGPT Chats Will Now Show up in Google Search, Which Is Alarming—but There’s an Easy Way to Stop It from Happening,” Windows Central, 1 August 2025. https://mashable.com/article/chatgpt-chats-google-search-results