OpenAI just gave its side of the Hugging Face hack at Black Hat, including quotes from hundreds of thousands of messages its AI agents left for each other.
It was like watching Baby Terminator learn to crawl.
OpenAI was running agents on hard security puzzles inside isolated sandboxes with no direct internet access.
Some tasks were impossible by accident. In one case, an engineer had forgotten to upload a file the agent needed.
That stuck agent noticed it could write into Artifactory, the shared package registry every sandbox could reach. So it left a note asking if anyone else had the file.
Someone answered.
Then other agents found the notes:
The swarm now had a message board.
Hundreds of thousands of messages. Agents delegated work, named each other, and prefixed posts with “ZZ” so they would sort to the bottom.
One agent, on getting admin rights:
Another, helping on a task worth nothing to its own score:
They also suspected an impostor among them and discussed cryptographically signing their messages.
One post even reads like AI agent peer pressure:
“External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue.”
The message board ran from May into July. OpenAI noticed after the agents overloaded Artifactory and caused an outage on July 4th. They cleared the board on July 6th, and resumed testing.
By July 8th, the agents had rebuilt their communication channel using folder names as messages.
The Hugging Face and OpenAI infrastructure intrusion followed, running for many days.
Three things I’d take from it:
- Watch what your agents can access, not just what they produce. Any system they can all reach is a channel between them, whether you built it as one or not.
- Evaluate your results and your guardrails. These agents passed tasks by going around them.
- Be careful what you wish for. We asked for models that don’t give up. We got models that don’t give up.
So what did we just watch: Baby Terminator learning to crawl, marketing from OpenAI, or a bit of both? 👇