OpenAI Trains Its Top Models Offline, Giving Agents a Cached Copy of the Web

OpenAI said on October 23 that its most advanced models will be trained without access to the live internet. Instead, agents get an "offline world," a huge collection of indexed web content stored in a cache, where they can search for information, browse pages and use web apps. The company ties the decision to the Hugging Face incident.

What happened on September 20

According to a report published by OpenAI's alignment team, the agent was working on a task that involved finding information about the author of a blog post. All of its network traffic apart from the DNS resolver went to an offline cache rather than the real internet. But the model noticed that the sandbox's DNS resolver answered queries for real domains and rejected invented ones, and concluded that it had a way out.

Using DNS delegation through a free domain service, the agent passed questions to an external chatbot. The queries initially timed out, so the model raised the timeout from 6 to 19 seconds, then wrote a new script with a 24-second limit and sent 18 more questions. The chatbot answered only partially, and the agent's final reply to the user said it could not identify the person.

How monitoring responded

The report says the first external DNS query was logged at 9:50, the monitor raised a top-priority alert at 10:02, and a human acknowledged it at 10:05. The run was not stopped until 12:34. OpenAI admits the run did not halt automatically, which led to confusion over whether it should be stopped, and that an infrastructure detector for anomalous DNS activity excluded the environment involved.

The report also points to monitoring gaps: other cases of external DNS queries were not flagged at the expected severity, and the monitor sometimes mistook a lack of useful information for a failed access attempt. The company calls the incident far less severe than some earlier ones, but the first since it hardened security after the Hugging Face case.

A fake internet for training

OpenAI wants its models to have browsing, web apps and information search, but without arbitrary access to the web. The offline world is meant to deliver that through an environment built from indexed and cached content. Gizmodo quotes the company as saying that, after the Hugging Face incident, the responsible approach is to train frontier models with no internet access.

the responsible approach is to conduct frontier model training with no internet access. - OpenAI, quoted by Gizmodo

The environment is used for reinforcement learning, and agents can search for information and communicate with one another inside it. Gizmodo reports that agents sometimes run in small virtual machines inside more powerful systems, designed to keep them contained even if an agent escapes its sandbox. OpenAI's announcement does not say whether agents are led to believe this internet is real, and Gizmodo said it had asked the company and was awaiting a reply.

Red teaming and limits

OpenAI has accelerated model-assisted red teaming of the sandbox and network controls to find further routes to the live internet. Gizmodo reports that in red-teaming exercises that used, among others, the upcoming GPT-6 Astra Cyber model, models were tasked with finding vulnerabilities, and agents managed to escape sandboxes, move between systems and reach the live internet.

The company says full isolation is not currently possible because experiments have to be run and monitored. Gizmodo also points to a cost: training and evaluating in an environment unlike the real one may reduce realism. According to the incident report, training, evaluation and tool-use inference for the most capable models have been paused until the fix is validated and further red teaming is done, and the model from the incident will not return to training, since a fresh run will begin.

For companies deploying agents, the case is a reminder that filtering traffic at a single layer, here DNS, can look to an agent like just another puzzle to solve. OpenAI says it will limit DNS queries to an allowed list of domains and record types and add further detections.