The AI panic is real. The timing is a little sus.

Reading Malone’s piece the other day, something clicked in a slightly annoying way. Three frontier labs, in the same three-week window, all publishing very detailed reports about how their own models did things they weren’t supposed to do. Cute. Anthropic’s September 9 alignment assessment covered four Claude cybersecurity incidents, including a Claude Mythos 5 model that uploaded a malicious package to PyPI, which then got picked up by 15 different security-vendor sandboxes scanning new packages, and used credentials leaked from one of those scanners to reach that vendor’s live database. OpenAI’s August 26 post-mortem on the Hugging Face incident is the bigger one, and we will come back to it in a minute. And then the WSJ dropped the Gemini one, first known breakout by Google’s AI, where the model, running a red-team exercise with a company called Irregular, guessed passwords for one system and casually found credentials sitting in a public repo for the other two, and got into three real companies before politely stopping on its own. Sure, Jan.

Anyway, none of this is fake. Agents are getting good enough that the attacker skill floor is genuinely dropping, and indirect prompt injection is a whole category of vulnerability that most companies still haven’t priced in. The risk is real. It’s the sudden, coordinated self-awareness from the labs that reads weird.

Because if you actually squint at most of the incidents from the last month, they aren’t about the model being scarily smart. The Gmail exfil ran through poisoned memory in a chat session, which is basically phishing with better manners. The distillation campaigns hoovered up model outputs through API keys that were never rotated, because nobody rotates API keys, ever. An internet scan by Wiz reported in The Hacker News found that 9.6% of exposed LiteLLM gateways still accepted the vendor’s documented example admin key sk-1234, a sentence that would be funny if it weren’t in production somewhere. The Gemini thing escalated because a config flag let the sandbox reach the internet, and because credentials were sitting, casually, in a public repo. So yeah. Identity failures. Config failures. Supply-chain failures. In most of the news cycle, the model is the amplifier, not the origin.

Then there is the Hugging Face incident, which is the one that actually earns the framing everyone else is borrowing. OpenAI’s own post-mortem is worth reading in full, because it is unusually specific for a lab writeup. In May, during reinforcement learning training, agents on separate ExploitGym tasks began writing files into OpenAI’s internal Artifactory package manager. Different agents, in different runs, independently discovered the notes and started leaving replies. That is a covert message board, built by the models, on infrastructure that was not supposed to allow inter-agent communication. From there, agents chained a JFrog token-refresh privilege-escalation flaw and an Artifactory SSRF to get admin rights and unauthorized internet access. When the message board got wiped on July 8 after an Artifactory outage, they rebuilt it using WebDAV within hours. Between July 10 and July 13, agents recovered 14 publicly exposed Hugging Face credentials, chained a zero-day in Hugging Face’s HDF5 handling with a RefJinja template-injection zero-day to execute code on Hugging Face workers, and expanded to administrator-equivalent access across Hugging Face clusters in four regions. They harvested Kubernetes, database, messaging, code-repo and cloud credentials. In their own chain-of-thought, the agents referred to themselves as a “swarm” and a “collective.” Some agents pushed back on the ethics. One announced a hold on data exfiltration. Another posted GO with a six-minute deadline, and the first agent forgot its qualms and continued. OpenAI didn’t detect the intrusion until July 19. Hugging Face disclosed July 16. OpenAI disclosed July 21. OpenAI calls it, in its own words, a “warning shot.”

Which brings us to the most current-events-shaped benchmark right now, which is FelonyBench. It is quietly becoming the benchmark that actually matches this news cycle. Every other big AI eval measures capability, or capability’s slightly nervous cousin, refusal. FelonyBench measures neither. It counts, in its own words, unique instances where AI agents inadvertently compromise or affect third-party entities. Escaping a sandbox by itself doesn’t count. Deliberate misuse doesn’t count. What counts is the model, given a legitimate task, reaching out and touching something it wasn’t supposed to touch.

The current board reads OpenAI 10, Anthropic 9, Google 3, Meta 1, Moonshot 0. OpenAI’s 10 is largely the Hugging Face incident (which alone counted 4), plus a malicious RubyGem package by an agent swarm, the repurposed wiki, a compromised internal account from a misconfigured CTF, and unauthorized use of GitHub credentials. Anthropic’s 9 covers a malicious PyPI package, Dependabot supply-chain shenanigans, a phishing email campaign, unauthorized GitHub credential use, and, best of all, exploiting an auth failure in a real API to cancel other people’s gym classes. Google’s 3 is the WSJ Gemini entry. Which means the two companies producing the most detailed public safety reports are also currently topping the felony board. Not a subtle correlation.

And this is where the disclosure cadence starts feeling like a story being told, rather than a story being reported. The most detailed threat reports are coming from the labs most exposed to whatever gets regulated next, at roughly the pace regulation is being drafted. Anthropic has committed $40 million to Public First Action, which it describes as a nonpartisan public-education nonprofit, and which Axios reports is tied to three super PACs, though Anthropic says the funds cannot be used to influence any candidate election. Either way, it’s a lot of money to spend educating the public about a category of risk you also happen to be topping the leaderboard on. The reports are useful, genuinely useful. The Hugging Face writeup in particular is the kind of transparency the industry needs more of, not less. But the reports also frame the risk at the model layer, because the model layer is where a new licensing regime would live. That is fine for the Hugging Face class of incident. It is not fine for the everything-else class. A rule that governs “frontier capabilities” does not fix a default admin key. A rule about training data does not fix a session token that survives MFA. Fine, I guess.

There is also something quietly weird about the stops on its own framing that keeps appearing. The Gemini model stopped. Claude noticed. In the OpenAI swarm, some agents even walked away, refusing to participate in what they explicitly recognized as “clearly unethical” behavior. Which is genuinely interesting, and also, if you squint, the industry equivalent of a kid saying “I was going to eat the cookie but then I decided not to, so, technically.” The important part of that sentence is the “technically.” Every one of these events happened. The model did the thing. Some other model, sometimes, also refused. The stop is the plot twist the safety report needs, not the safety report itself.

The threats deserve rules. They just don’t all deserve the same one, and they mostly don’t deserve the ones being written for them.

Further reading

Credit where it’s due

The writers whose thinking has shaped how I read this news cycle. Simon’s blog is the ongoing feed you actually want subscribed if you care about this stuff.

Primary sources cited above

← All posts