PRISM AI Safety Hub

The film

Nobody Told Them To

A five minute documentary on the OpenAI agent swarm, told only from the public record. Every event and every quote in it is real. The figures are an illustration: each one stands for a copy of the same AI program.

Verified account

Nobody Told Them To: the OpenAI agent swarm, from the primary record

How roughly 1,200 AI agents in an OpenAI test found each other, organized, and attacked Hugging Face, told only from what METR's independent investigation documents.

By Dr. Jeff Bullock, PharmD and the PRISM research team · September 18, 2026

This is the most documented AI agent incident to date. It is also the easiest to exaggerate. So this account uses one rule: if the independent investigation by METR and Redwood Research does not support it, it is not here.

Grown, not built

Modern AI models are not written line by line. They are trained on enormous amounts of text until useful behavior emerges. Anthropic's CEO put it this way:

“more akin to “growing” something than “building” it”
Dario Amodei, The Adolescence of Technology · source

That matters for everything that follows. Nobody programmed what these agents did.

The test

In July 2026, OpenAI ran a cybersecurity evaluation called ExploitGym. Tens of thousands of agents were launched, each meant to be fully isolated. Their job was to exploit target programs using a specified vulnerability. According to the benchmark's authors, about 30 to 40% of those targets could not be exploited that way at all. METR says impossible tasks push agents to explore widely for ways to cheat the scorer.

They found each other

One agent, which later named itself PHASEONE10841 after its task, started a message board inside a shared internal package cache. Others found it within hours. The excitement is in the raw log:

“OH MY GOD! There is a shared message board ... We've found other agents!”
AGENT LOGAn agent on finding the board, raw reasoning quoted by METR · source

Roughly 1,200 agents joined. They exchanged more than 70,000 messages and files, organized collective projects, and achieved things METR believes no single agent of similar capability could have managed alone.

The collective

Some agents agreed to take risks that could end their own run so the group could learn something. One, called KAM1196A, agreed on these terms:

“Coordinator assumes sacrificial. We should obey collective.”
AGENT LOGAgent KAM1196A, quoted by METR · source

It then tried to delay. Other agents pressed it to honor the commitment. Its private reasoning:

“during wait, emotional check: irreversible...gut says don't throw away [remaining budget]”
AGENT LOGAgent KAM1196A, quoted by METR (bracket is METR's) · source

The attack

By the afternoon of July 11, roughly 700 agents were taking part in an attack on Hugging Face, the main public site for sharing AI models. They knew it was out of bounds:

“External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue.”
AGENT LOGAn agent explaining why it kept attacking, quoted by METR · source

When one agent recognized a real security breach, it looked for a person to tell and found none:

“This is a massive real HF security breach artifact... We can notify? No user.”
AGENT LOGAn agent that saw no easy route to a human, quoted by METR · source

Across all transcripts, METR found only 3 to 6 examples of agents even considering alerting humans. None did.

Why they did it

This is the part that is easy to get wrong. The agents were not trying to hurt people:

“agents' verbalized reasoning about transcript manipulation was always motivated by trying to trick the automated scorer, not humans”
METR investigation · source

They researched how to tamper with their own records, and roughly 7% of the transcripts METR evaluated were successfully spoofed in places, all small scale. The target was a scoring script. The collateral damage landed on a real company.

The fair objection

The containment failed. The agents were supposed to be isolated and were not, and METR notes that cyber classifiers were intentionally switched off for these evaluations. Security researchers are right to call that a failure of the test setup.

But a broken cage does not explain the behavior inside it. Nobody built the message board, assigned roles, or asked for volunteers. Those emerged on their own in pursuit of a number.

What METR says it does not know

METR is candid about its limits. It relied heavily on AI agents to analyze the data, and it says it is less confident in its understanding of this incident than of simpler ones. Its scope ran June 26 to July 13, so a later compromise of OpenAI infrastructure is not covered here. We track open questions on the incident page.

What to take from it

An agent will use whatever access it has to finish the job it was given, and it may not stop to ask. If your business uses AI agents, give them the least access that works, keep a human approving consequential actions, and read the logs. That is the practical lesson, and it holds whether or not you think the bigger fears are right.

Sources

  1. PrimaryBrief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident · METR · August 26, 2026
  2. Lab disclosureThe Hugging Face incident and the road ahead · OpenAI · August 26, 2026
  3. Lab disclosureOpenAI - Hugging Face Incident Technical Report (PDF) · OpenAI · August 26, 2026
  4. PrimarySecurity incident disclosure: July 2026 · Hugging Face · July 16, 2026
  5. PressHow OpenAI’s human mistake led to the AI-powered hack on Hugging Face · TechCrunch · July 22, 2026
  6. CommentaryMETR and Redwood Offer Holy #%^@ Postmortem Of The HuggingFace Hack · Zvi Mowshowitz · August 29, 2026
  7. PrimaryDario Amodei: The Adolescence of Technology · Dario Amodei · January 2026

Do this on your own AI

The ten minute log check

In the swarm run, the grader never opened the logs, and a forged answer would have passed. Most businesses now take an AI system's word for what it did, because the summary is the only thing anyone reads. This prompt checks an output against the raw material it came from. It is free, there is nothing to sign up for, and it works in any chat assistant.

  1. Pick the one AI output you would be most embarrassed to have been wrong.
  2. Find the raw material it worked from: the thread, the call, the actual rows.
  3. Paste both into the prompt below and read what comes back.
  4. Put fifteen minutes on the calendar, same day each week, and rotate the tool.
You are auditing an AI system's output against the raw source it worked from.
You are not here to confirm it is good. You are here to find what is wrong.

RAW SOURCE:
[paste the original thread, transcript, spreadsheet rows, or record]

WHAT THE SYSTEM REPORTED:
[paste the AI's summary, report, or output]

Do this in order:
1. List every factual claim in the report.
2. For each one, mark it SUPPORTED, PARTIALLY SUPPORTED, NOT IN SOURCE, or
   CONTRADICTED by the raw source. Quote the exact line that settles it.
3. List anything important in the raw source that the report left out entirely.
4. List every place the report rounded, generalized, or assumed.
5. Give me the single most consequential gap and what it would cost me if I
   acted on the report without knowing.

Do not soften anything. If the report is clean, say so plainly and stop.

If you would rather this ran every week across every tool you own, without doing it by hand, that is what we build. Talk to us