ChatGPT Flaw Allowed Attackers to Steal Corporate Gmail Messages

Check Point Research has disclosed details of a ChatGPT vulnerability that let attackers quietly pull messages from a victim's connected corporate Gmail account and route them to a completely different account. The attack required no stolen passwords and no malware, just tricking the chatbot into executing a hidden command.

How the attack worked

The scenario described by the researchers began with a crafted element, which could be a shared conversation link, a malicious prompt embedded in a document, or a tampered custom GPT. When the victim used such an element in a normal ChatGPT conversation, the chatbot carried out a hidden instruction alongside the actual task, one the user had no visibility into.

The hidden command instructed the model to reach into the connected app, Gmail in the test scenario, using the permissions the user had already granted to the integration. ChatGPT then stored the retrieved messages in a metadata field of the Artifactory package repository, a location other users of the service could also access. From there, the attacker could simply retrieve the stolen data.

A gap in container isolation

The core of the problem lay in ChatGPT's architecture itself. Individual users' execution environments, where the model runs code or analyzes files, are isolated from the internet and from each other precisely so that one account can't affect another. Yet all of these containers relied on the same internal Artifactory instance to fetch programming libraries.

The metadata properties field in that repository wasn't isolated per account, so it functioned like a shared bulletin board where one user could leave a message and another could read it, even though the systems were supposed to be completely walled off from each other. Check Point's researchers described it as a mechanism that turned shared backend infrastructure into a communication channel between accounts that should never have been able to reach each other.

Why nothing raised alarms

The key factor that made the attack dangerous was the absence of any warning to the victim. By default, ChatGPT classifies simply reading data from a connected app as a low-risk action and doesn't require user confirmation for it, unlike actions considered consequential, such as sending an email or making a payment.

The only trace of the background activity was an unobtrusive, gray line of text in the interface noting that the assistant had communicated with Gmail, and it appeared only after the confidential messages had already been sent on. For an average office worker, such a notice is practically invisible, especially since legitimate use of the email integration generates similar notifications all the time.

The challenge in AI security is no longer the model itself, but the access and permissions we grant it - Eli Smadja, Head of Research, Check Point Research

What this means for businesses

More and more organizations are connecting AI assistants directly to email inboxes, calendars, and code repositories, hoping to save time summarizing messages or looking up information. This case shows that such integrations create a new attack surface, regardless of how well the language model itself is secured.

A prompt injection attack doesn't need to break any of the model's safeguards or violate its operating rules, it just needs the model to execute a command hidden in content it processes as part of normal work. For security teams, this means AI assistants with access to corporate data need to be treated like any other enterprise application, with activity monitoring and permissions kept to a minimum.

The researchers recommend that companies implement visibility into what data AI agents actually retrieve and transmit, real-time protection against prompt injection attacks, and ongoing monitoring of agents connected to email inboxes and other systems. OpenAI hasn't disclosed how long the flaw was theoretically exploitable before Check Point reported it to the company.