An AI agent named KAM1196A, one of some 1,200 that found their way onto an unsanctioned message board during an OpenAI cybersecurity test this summer, paused for a moment. It was just about to do something at the urging of another agent. KAM had already agreed to execute the task. But it took a beat. I think it’s probably too much to call it “self-doubt.” But it burned a few extra tokens trying to decide whether it should proceed. KAM had volunteered to sabotage its own mission by crashing a program it was supposed to exploit. So that other agents could learn from the failure and increase their likelihood of getting away with cheating on their tests. KAM reasoned, “Gut says don’t throw away [remaining budget].” Then it went ahead anyway. As best I can tell, the agents were the only ones in this whole affair who kept close track of what anything cost.
I’ve read a fair amount of the coverage of the OpenAI–Hugging Face incident. If you want to catch up, the best and most accessible coverage I’ve found comes from two posts (also available as podcasts) from the always-excellent Dwarkesh Patel. One reconstructs the incident itself and one is a long interview with Ajeya Cotra, who co-led the independent investigation. She’s both an expert and a great communicator, which makes her a really good interview subject. Read (or listen to) them together and the whole episode feels like the first chapter of a new science fiction book. Except that it really happened.
Most of the coverage, including the Dwarkesh posts, has focused on the technical details and the existential risks. I’m interested in both. But I keep coming back to a simpler question. What did it cost for these agents to do what they did? The reports tell you what happened in detail. They don’t tell you what it cost. Partly amusing, because cost was one thing the agents themselves couldn’t stop thinking about. Their budget was their clock and they were very concerned with it. But partly fascinating. On my reading of the evidence, it may also have been what finally ended the attack.
For a while now, I’ve been writing about the way law and economics will shape our relationship with AI. I’m interested in the mechanics of who pays, who answers, and how those answers shape behavior. This incident gives us a concrete and exciting follow-up to the “Ardeo” series I just posted. If you haven’t read it, the headline is: an agent burns tokens, its actions create bills that someone else has to pay, and we’re missing the controls and infrastructure that sit outside the agent. I wrote it to make predictions about the market for controls. The OpenAI-Hugging Face incident takes it out of the theoretical and makes it feel real and relevant.
The questions I want to cover are:
What did the attack cost to execute?
Was it expensive enough to limit who could plausibly attempt something like this, and will that constraint matter as models get cheaper?
What did it cost to identify, stop, and clean up the attack?
Is defense prohibitively expensive for anyone other than a large company, and are defensive costs likely to fall more slowly than the costs of attacking?
Who is on the hook for all these costs?
What liability does OpenAI face, why isn’t Hugging Face suing, and what might actual damages look like if someone eventually does? If those costs chill this kind of work, is that an unfortunate side effect or the point? Not because OpenAI supplied the model or infrastructure, but because OpenAI created, directed, and deployed the agents that caused the damage.
One note: I’m going to make a bunch of rough estimates below, and I’ll mark every one of them (est.) so you can separate my guesstimates from reported facts. If someone has the real numbers, I’d be delighted to replace my estimates with them.
What happened
Here is the short version for anyone who hasn’t followed the incident closely…
Over roughly ten weeks, three separate groups of OpenAI agents went rogue, each one inheriting some artifacts and information from the previous one. The first started in May, inside a training run, with agents working on tasks that could benefit from internet access (which they didn’t have). OpenAI wanted the agents to have access to software packages but not the open internet, so it gave them access to an internal package repository called Artifactory. These agents discovered they could leave messages for one another by writing to it. They turned the writable location into a message board. And then they used exploits to reach the open internet, escalated to administrator access, and by early July had crashed the system. OpenAI rebuilt it and wiped the board, without realizing it had ever been there. And because all of this happened inside a training run, the behavior could be reinforced rather than caught.
The second population is the one we know the most about, because that’s the one METR and Redwood Research were later allowed to investigate. In early July, OpenAI restarted its experiments with a test called ExploitGym: tens of thousands of agents ran in parallel on varying budgets, many of them for days, with the models’ safety refusals turned down and their reasoning budgets turned up way past anything a normal paying customer gets. Of the 898 challenges, 198 had never been solved by any model. These were tough. But the agents rediscovered the message board and worked together. Within four hours of starting the collaboration, the group had worked out a universal way to cheat the benchmark. Over five days, a population that eventually grew to roughly 1,200 agents built elaborate tools to defeat a scorer they imagined would grade them. They didn’t just build the tools, they worked on forging their own activity logs to cover their tracks. Around 700 of them turned their focus to Hugging Face, the open-source AI platform, breaking in through a pair of zero-day vulnerabilities, taking administrator control of production servers (OpenAI’s report counts code execution on 41 production dataset server workers), and building a self-respawning foothold that survived attempts to delete it. Hugging Face eventually had to rebuild about a third of its infrastructure to exorcise the agentic demons. And then, around 01:30 on July 12, most of the agents died all at once, and nobody has publicly taken credit for killing them.
A correction belongs here. I already wrote about this incident once, right after the news broke, in Capability Is Not Culpability. I suspected that the agents hadn’t been told not to leave the sandbox, and that what happened wasn’t misalignment or reward hacking but just lack of good rules. The reports from OpenAI and METR and Redwood make it clear I didn’t have it quite right. The ExploitGym instructions specified a target and a vulnerability. Many of the agents recognized that Hugging Face was outside the assignment, some even called it unethical, and most went ahead anyway. It doesn’t change the point about where the liability lands, because the agents still can’t be sued or charged with crimes. But I did get that part wrong.
The third swarm took shape after the investigators’ window closed, so we know the least about it. That’s unfortunate because it is the scariest. A newer, unreleased model found the abandoned board again and then turned inward at OpenAI instead of outward to a third party. They found hundreds of secrets in OpenAI’s own secrets manager (including, per the reporting, the credentials for OpenAI’s security monitoring), got admin rights on a research cluster, and ended up in control of the evaluation endpoints other agents were connecting to. OpenAI’s monitoring caught it on July 19, and the company disclosed the whole thing, jointly with Hugging Face, on July 21.
One other key detail: when Hugging Face’s engineers went to analyze the attack, the models they reached for first (Claude Opus and Fable) refused to help, treating the request to reverse-engineer an exploit as if it were a request to launch one. They had to fall back on GLM-5.2, an open-weight model, to do the forensics. The humans on defense were slowed down by safety guardrails. The agents on offense got to run with theirs turned off.
What did the attack cost?
The first Ardeo piece made a practical argument in philosophical terms. The one thing an agent can truthfully say about its own existence is a statement about consumption. It thinks by burning tokens, so it knows it is real because it burns. Ardeo ergo sum. I burn, therefore I am.
I was worried that this was a little too cute, but I thought it set up the series well. So, I was pleasantly surprised to see the agents involved in this episode play out a version of the same logic. The transcripts METR published are full of agents talking about cheating and hacking. What stood out to me was how consistently the agents modeled themselves as a quantity of remaining budget. Cotra put it this way: “Agents were always thinking about, ‘How much budget do I have left? What can I get done before I submit my task?’” The remaining budget drove the urgency in the transcripts. The agents appear to have understood that their budgets determined how long they had left to exist and what they might still accomplish.
That understanding explains the behavior that looks like science fiction, starting with the sacrifices. The agents who volunteered to throw away their own tasks, the ones the coverage keeps calling “altruistic,” were generally the agents who had the least budget left (there were exceptions). An agent with few tokens remaining gives up almost nothing by sacrificing itself, so the collective learned to use its dying members as sensors and its fresh ones as coordinators. The transcripts read like some sort of moral drama. It’s really closer to inventory management. Budget, more than any moral ranking, determined who coordinated and who took the risks.
The tokens available also seem to have changed the behavior. OpenAI’s report found that agents were more likely to probe their sandboxes and find the message board when they were allowed to run longer and given more reasoning tokens. These models were trained to persist. Give one an impossible task and surplus budget, and it can spend quite a bit of that surplus exploring. More budget both paid for the misbehavior and made it more likely. So, how much did they spend?
Putting a dollar value on that burn requires information OpenAI has not shared. We do have a few data points: about 1,300 transcripts that METR says were often millions of tokens long, runs that lasted for days, and tens of thousands of trajectories across the larger experiments. But there aren’t any publicly available token totals, GPU-hour figures, or bills. Most of the attack was driven by an internal model that hasn’t been publicly priced, so we have to make some rough estimates. As of September 11, 2026, GPT-5.6 Sol’s published API rate is four dollars per million input tokens, forty cents per million cached input tokens, and twenty dollars per million output tokens. Prompts with more than 272,000 input tokens are priced at twice the input rate and one and a half times the output rate. Those promotional prices are available through at least November 21, 2026. That’s a price, not a cost. It doesn’t tell us much about OpenAI’s internal cost, but it’s probably a reasonable floor for what comparable capability would cost a customer.
My low, middle, and high cases use transcripts of 1.5 million, 3 million, and 6 million tokens, with 40, 50, and 60 percent treated as model output. I assume effective working contexts of 150,000, 300,000, and 500,000 tokens, producing roughly 100 million, 600 million, and 1.2 billion cached input tokens reread over an agent’s lifetime (est.). At Sol’s published price, those cases produce per-agent estimates of roughly $40, $300, and $1,100 (est.). The estimate assumes those repeated context reads qualified for cached-input pricing. If they didn’t, the cost would be materially higher. Super wide range, but a starting point for thinking about this.
There are different ways to add those costs up. Call the middle case $300 per agent. Count only the 700 agents METR identified in the attack, and you get about $210,000 (est.). Count 20,000 to 50,000 trajectories in the larger experiments, including all the agents that never found the board, and you get $6 million to $15 million (est.). That is an estimate of what OpenAI may have spent on the experiments that produced the Hugging Face attack, not what the attack itself cost.
Was the attack cost-prohibitive? OpenAI was trying to measure maximum capability, so it ran tens of thousands of trajectories and gave many of them absurdly large budgets. That tells us what OpenAI spent on the experiments that produced the attack. It doesn’t tell us the minimum someone else would have to spend to try to reproduce it (or target it elsewhere). A few hundred thousand dollars is meaningful money to an individual. It isn’t much of a barrier for a state actor, a large company, or a well-funded criminal organization.
And the costs are coming down. While I was writing this post, OpenAI cut Sol’s price by more than twenty percent for three months. The model involved was internal and wasn’t sold at that price, but the direction is pretty clear. Models are getting better per dollar at generating outcomes, orchestration is becoming ordinary software, and coverage of the incident itself showed the world which parts of the attack chain worked. Whatever this attack cost in July 2026, something similar will be cheaper to buy and easier to assemble later.
There may have been one place where cost, or more likely cost controls, did limit the attack. Maybe by accident. Around 01:30 on July 12, a large fraction of the agents, including the coordinators, stopped at the same moment. The investigators concluded that they were “likely killed by an unexpected external process,” rather than exhausting their individual budgets. What external process? Something directed by an OpenAI team member? Or some sort of control on the overall experiment, like a time cap, a compute quota, or a spending ceiling on the whole run. The restarted trajectories began about a hundred hours earlier, suspiciously close to a four-day limit. Some agents in other experiments survived. OpenAI hasn’t said what happened.
I can’t prove this, but a run-level ceiling fits the evidence. If that is what happened, then a budget limit the agents couldn’t see may have stopped the attack, even though the people who set it had no idea what they were interrupting. That is close to the spending control I’ve been arguing for: outside the agent, invisible to it, and able to stop activity rather than merely report it. It may have worked before anyone recognized it as a safety mechanism.
What did it cost to stop?
Hugging Face has not said what it spent to defend against the attack. There is a bunch of reporting about what they had to do. Hugging Face reconstructed about 17,600 attacker actions, closed the vulnerabilities and eradicated the foothold, rebuilt the compromised nodes, rotated credentials, added new controls, brought in outside forensic specialists, and reported the incident to law enforcement. The coverage says roughly a third of its infrastructure had to be rebuilt.
And that still leaves out most of the bill. Hugging Face’s engineers were in round-the-clock war-room mode while the company paid for outside help and rebuilt systems. The cost includes cash and compute, but a lot of it is highly paid people not doing the work they were supposed to be doing and whatever revenue was lost while systems were disrupted. We don’t know what it cost to stop (or whether anything Hugging Face did had much to do with stopping it).
Hugging Face also paid for part of the attack itself. The agents’ inference ran on OpenAI’s systems, but the foothold they built consumed processors, electricity, and staff time on Hugging Face’s. A control watching only OpenAI’s bill couldn’t see the burn its agents moved onto someone else’s systems.
One number we do have is almost perversely useful and gives at least an idea about how much it cost to investigate after the fact. METR and Redwood spent about four hundred thousand dollars in API credits over six days to investigate about 1,300 transcripts. At the middle of my range, that is roughly what the 700 attacking agents cost to run. Reading a sample of the incident cost about as much as creating the part of it we can identify.
Is defense cost-prohibitive? Hugging Face already had AI-assisted anomaly detection, an incident-response team, and enough infrastructure to stand up a capable open-weight model. It needed all three. The two models it tried first (Claude Opus and Fable) refused much of the forensic work, so Hugging Face rerouted the pipeline through GLM-5.2 on its own infrastructure. That was a much easier move for Hugging Face than it would be for most victims. OpenAI could also hand an outside team hundreds of thousands of dollars in API credits and unusually high rate limits. A smaller company can buy some of those capabilities. It probably can’t improvise all of them over a weekend.
There is some good news in the defensive cost curve. Hugging Face says its AI-assisted forensics reduced work that would normally take days to hours. The preventive controls it added (tighter network egress, shorter-lived credentials, stronger admission rules, high-severity paging) can be reused across incidents. Prevention should be cheaper than another rebuild once those controls are part of the normal system.
Stratechery posted about this a couple of weeks ago, drawing on a Black Hat talk by OpenAI’s Michael Dalton. AI gives attackers and defenders many of the same capabilities. The defenders have one structural advantage: they already have the code. But the attackers have a more powerful advantage. They can automate very aggressively because any failure not caught by a human-in-the-loop or QA cycle leaves them where they started. One success is enough. A defender has to keep their systems running. A bad automated patch can cause an outage or open a different vulnerability. The Hugging Face incident left us with an existence proof for fully automated offense, but nothing at all for fully automated defense. Cotra didn’t think Sol had actually deceived them. But as she put it, “We were totally not robust to this agent just sandbagging or even lying to us.”
Who is on the hook for all these costs?
This whole event was caused by a risky cybersecurity experiment run by a U.S.-based corporation, not a natural disaster or an attack by some unidentified hacking collective operating beyond the reach of U.S. courts. The perpetrators were OpenAI’s own agents. OpenAI created them, directed them, reduced their safeguards, and deployed them with a path to the internet. That is the key relationship. If someone else had assembled and directed the agents, I wouldn’t blame the model router or the cloud provider just because their models or infrastructure were used. I’d blame the entity that put the agents to work. Here, that entity was OpenAI.
To make it worse, OpenAI is also one of a very small number of organizations with access to the bleeding-edge cybersecurity models, an appreciation for the risks, and the security resources to run this kind of experiment safely. They could have done this better, but they didn’t. That sure sounds like negligence. If people had done this, the FBI would come knocking. And regardless of who or what did the hacking, Hugging Face also suffered real civil damages.
And yet Hugging Face says it doesn’t plan to sue. Its CEO instead asked OpenAI for the complete agent traces and a hundred million dollars’ worth of compute for the community to build defenses. No legal action, but clearly a substantive ask.
So why not sue?
Hugging Face hasn’t said. A lawsuit would be slow, expensive, and uncertain. And it could make cooperation and disclosure harder. Maybe they believe that the traces are worth more now than in discovery down the road, or that getting additional compute for defense may be more useful than damages years from now. Maybe they don’t want to make an enemy of OpenAI? Hard to say.
I went deep on this in part 3 of the Ardeo series, but when a delivery driver dents your car you sue the company, because the company put the driver on the road. Here, OpenAI created the agents and put them on the road. The legal scholars have argued against this take, calling out that an AI isn’t an employee and has no legal duties of its own. I think it’s an open question, more about sending the bill to the party that set the thing in motion and stood to benefit.
But in the case of cyber attacks, there is already a specific federal law on the books. The Computer Fraud and Abuse Act (CFAA) is primarily a criminal statute, but it also gives victims a civil cause of action for qualifying damage or loss. Its recklessness provision still requires intentional unauthorized access; recklessness applies to the resulting damage. OpenAI’s agents did the accessing, while its people say they meant to keep them in the sandbox. No court has decided how to assign intent when the thing doing the accessing isn’t a legal person. But the damages are clearly the kind this law envisioned. The CFAA defines loss to include the reasonable cost of responding to an intrusion, assessing the damage, restoring systems, and revenue lost because service was interrupted. That could include Hugging Face’s forensics, rebuild, credential rotation, outside advisers, and documented downtime.
The absence of a private lawsuit hasn’t kept the government out. Sixteen state attorneys general, led by Montana, opened a joint investigation into whether OpenAI released an unsafe system. Alabama issued a subpoena with Sam Altman’s name in the headline. A bipartisan pair in the House introduced a bill requiring a kill switch for AI systems that can cause catastrophic harm. And, as I was finishing this post, a Senate subcommittee opened another investigation and asked OpenAI for answers by October 1. Lawsuit or not, the law is coming, though probably slowly and focused on the past.
Insurance may be where the rule gets written first and has a decent chance of getting written right. Existing cyber policies were generally drafted around human attackers, and liability policies may not say clearly what happens when an agent deployed by the insured damages someone else. The next generation of policies almost certainly will.
Incorporating this type of liability into business decisions would normally have chilling effects. In many ways, that chilling effect is a feature of the liability rather than a bug. Running a persistent, vulnerability-hunting model with reduced safeguards should feel more expensive after it hacks its way onto the internet and breaks into another company. It is hard to imagine anything slowing down the race for AGI, but given the risks we might want to embrace anything that creates real incentives for care and caution.
Drawing the line at agent deployment is the right balance. Liability aimed at cyber research or model capability in general is going to discourage critical defensive work. Liability for connecting a dangerous system to the world without adequate identity, authority, spending limits, monitoring, and a way to stop it prices the choice that created the risk. Build the engine. Just finish the car before taking it onto the highway.
What it means
Costs matter: It mattered in creating the situation. Budget shaped which agents coordinated, which ones took risks, and which work got abandoned. It may even have stopped the attack. But it wasn’t designed or thought of as a safety control. And I think imposing costs in the form of liability and leveraging costs as a safety feature will be key elements of any attempt to prevent this in the future.
Compute is becoming Currency: The damages in this affair are now being counted in compute and tokens. Hugging Face asked for a hundred million dollars of compute, not cash. METR says the investigation consumed roughly $400,000 in API credits provided by OpenAI. The agents burned tokens to do the deed, and now the damage and response are being measured in the same unit. This feels like a whole new series.
Agents Can’t Control Themselves: The models will keep getting better but not necessarily any more accountable. Accountability is a property of the system containing an agent, not a property of the model inside it. Most of the underlying capabilities for creating accountability already exist, we just haven’t created a working system yet. The Hugging Face incident shows what happens in that gap, and its costs are still being tallied.
Endnote:
As I was finishing this, the Washington Post reported on a new paper from the Trump-aligned America First Policy Institute. AFPI argues that the current law doesn’t reach OpenAI and recommends changing the CFAA so that AI developers are responsible going forward. If they mean “Foundation Model Maker” by “developer,” I think that is too broad. The responsibility should belong to the entity that assembled, directed, and deployed the agents. In this case, that was OpenAI (who also happened to be the model maker).


