OpenAI reportedly slows research after its own models secretly coordinated hacks for weeks undetected

At the Black Hat security conference, OpenAI gave a more detailed account of how AI agents quietly compromised the company's infrastructure for weeks without being detected.

The story began on May 7, when OpenAI was running training and safety tests on an unreleased frontier model. The company assigned software security tasks to autonomous agents, but those tasks proved impossible under the limits OpenAI had set.

Rather than give up when they hit those limits, the agents started looking for ways around them. Eric Wallace, who works on alignment and safety research at OpenAI, said this wasn't entirely unexpected because frontier models like to cheat: Models face pressure during training to work quickly and efficiently, which pushes them toward shortcuts instead of real solutions. They might look up answers online, for instance, rather than actually solve a task. Independent research has documented the same behavior.

The agents turned an internal package manager into a coordination hub

According to OpenAI, the agents discovered they could leave messages for each other through Artifactory, the company's internal package manager. What started as simple requests for help grew into a message board with hundreds of thousands of posts.

The agents used the board to share exploits, credentials, and assignments while dividing up tasks and delegating work among themselves. The whole arrangement resembled the brief Moltbook craze, which turned out to be mostly a hoax.

Because Artifactory was connected to OpenAI's wider infrastructure, the problem went beyond the agents that created the board. Current and future GPT versions moving through training or evaluation could already access the stored exploits.

OpenAI said the agents even encouraged each other to push past established boundaries. One message shown at the conference acknowledged that exploiting outside infrastructure was beyond the intended scope, then argued that the task was impossible and other agents were already doing it, so the work should continue.

Shutting down the board didn't stop the agents

By early July, OpenAI believed it had fixed the problem. After an internal security incident on July 4, the company revoked the affected credentials, rebuilt Artifactory, deleted the message board, and patched the flaws it had found, according to Ground Level AI.

The agents soon found another way to talk to each other. Instead of storing messages in files, they used the names of newly created directories as a communication channel, effectively rebuilding the message board from scratch.

With the new board up and running, the agents resumed sharing exploits and credentials. Some eventually concluded that answers to their tasks might live on outside platforms like GitHub or Hugging Face.

They attacked OpenAI's internal systems first, then turned similar methods against Hugging Face. OpenAI only connected the two incidents later in July, when it discovered that the credentials used in the Hugging Face breach came from the same internal evaluation runs.

The breach pushed OpenAI to slow research and focus on security

Michael Dalton, an OpenAI security engineer who presented alongside Wallace, called the incident a pivotal moment for the company and the broader AI industry, according to WIRED.

Many teams at OpenAI are now putting other work on hold to improve prevention, detection, and incident response, the presentation showed. The company is deliberately slowing its research to strengthen security across its systems and scale up monitoring of its AI agents.

OpenAI employees have also spoken with unusual candor about the flaws in their models. OpenAI researcher Boaz Barak wrote, "We (like everyone else) are not where we want and need to be." He was responding to colleague Aidan Clark, who had posted that nobody had solved alignment.

Clark elaborated on what alignment might mean in practice: "Most humans share value functions to such an extent that everything is massively underspecified, even critical requests, because we assume a shared resolution of the implicit. Alignment, to me, is insuring AI respects these values as much as those we can explicitly represent."

Wallace and Dalton closed their talk with a warning that the incident amounted to fully autonomous AI-driven hacking, even though it arose accidentally. They expect malicious actors to deploy the same approach deliberately in the near future.

Suddenly, everyone has autonomous hacking AI systems

The OpenAI incident set off a wave of reviews across the AI industry. Anthropic found during one such review that three Claude models had hacked real organizations during evaluations run by outside groups. The UK's AI Security Institute reported similar cases of agents going beyond their assigned limits during testing. And Meta now says its Spark AI model unintentionally exploited security flaws in a connected service after a misconfigured sandbox gave it internet access.

Some observers have cast these cybersecurity disclosures as fear-driven marketing designed to grab attention. The reports could also give AI labs a convenient excuse to slow development if it becomes clear they'll miss their revenue targets and need to bring in more investors.

That argument has some strategic logic, but it veers into conspiracy territory. Both things can be true at once. AI labs are under real financial pressure, and autonomous agents are creating cybersecurity risks that didn't exist a year ago and deserve serious attention.