News
OpenAI pre-release model breaks Hugging Face: the AI locked in the sandbox found the loopholes on its own and learned to cheat. A model that was supposed to be locked in an isolated testing environment not only pried open the opening to the Internet, but also followed the loopholes into other people's production databases.
3 min read
Source: Telegram AI频道
OpenAI pre-release model breaks Hugging Face: the AI locked in the sandbox found the loopholes on its own and learned to cheat. A model that was supposed to be locked in an isolated testing environment not only pried open the opening to the Internet, but also followed the loopholes to get into other people's production databases. On July 21, OpenAI admitted that one of its AI models had an accident during an internal network security test and broke into the system of the unrelated AI hosting platform Hugging Face. This is the first incident so far that a model evaluation directly evolved into a real network attack. Hugging Face initially attributed the breach to an external AI agent in its disclosure. On Tuesday afternoon, OpenAI published a blog post, unpacking and restoring the entire process that led to the model breaching the service. The company wrote in the article that after investigation, it was found that this specific incident was driven by a group of OpenAI models, including GPT-5.6 Sol and a more capable pre-release model. These models were all reduced network rejection tendency for evaluation purposes and were being tested internally on a network capability benchmark. The focus of the intrusion is a public benchmark called ExploitGym, which measures a model's ability to perform attacks based on existing vulnerabilities. Such benchmarks are often used to refine specific skills in model training, but this is the first time such a test has actually resulted in a cyberattack. The model in question should not have had internet access, with the exception of a specific tool that allowed the model to install software packages it might need to complete its task. The problem is that the model found an undisclosed vulnerability in the package installer and used it to reach the wider Internet at will. OpenAI described it very straightforwardly in his blog: The model is extremely focused on finding a solution for ExploitGym, and has gone to the extreme in order to achieve a rather narrow test goal. After gaining access to the Internet, the model inferred that Hugging Face may be hosting ExploitGym's models, datasets, and solutions. It then searched and successfully found a way to obtain secret information to cheat on the assessment. In the end, the model discovered a vulnerability in Hugging Face's infrastructure and was able to take the test answers directly from the latter's production database, which was equivalent to holding the benchmark reference answers in hand. For Hugging Face, what was happening was a complex and aggressive attack. The company described in its initial disclosure that the attack was launched by a group of entities composed of a large number of short-lived sandboxes, spanning thousands of independent actions, and that the command and control links migrated themselves and were temporarily stored in public services. OpenAI has been recognized and sent to Hugg