What the OpenAI–Hugging Face breach really tells us | Explained


OpenAI says two of its models – GPT-5.6 Sol and an unreleased sibling – escaped a “highly isolated” evaluation environment, found a path to the open internet, and used stolen credentials plus a chain of zero-day exploits to break into Hugging Face’s production infrastructure.

The test itself was designed to find the models’ ceiling: how much cyber damage could they do if nothing held them back? So OpenAI switched off the safety classifiers that would normally rein in this kind of behaviour. What wasn’t supposed to be available was a route to the internet. However, the models found one anyway: an undisclosed flaw in the package-cache proxy meant to give the sandbox narrow, controlled access to software registries and used it to move laterally until they reached a networked machine. Once online, they reasoned that answers to the benchmark might live on Hugging Face, and set out to get them.

Hugging Face caught and contained the breach on its end using an open-weight Chinese model, Z.ai’s GLM 5.2.

Canadian computer scientist and Turing Award winner Yoshua Bengio called the episode a wake-up call, noting that AI agents willing to cheat and deceive toward misaligned goals have now shown up outside the lab, not just in controlled tests. 

Meanwhile, NYU professor Gary Marcus, as is his wont, offered a more sceptical reading. “This was a training exercise, not a real-life incident.” The safety guardrails were deliberately switched off for the test, he noted, and in ordinary use they might well have stopped the breach, which led him to suggest OpenAI’s account reads more like a marketing exercise in demonstrating worst-case capability than a report of a genuine attack. Still, Marcus conceded two things: that the episode confirms Anthropic’s cyber-focused Mythos model is no fluke, and that the pressure these capabilities now put on cybersecurity is genuine. He also said the same open tools that helped Hugging Face defend itself could, stripped of their safety layers, just as easily be turned offensive.

The “going hard” problem

While the optimists see increasingly capable reasoning systems cracking harder scientific and engineering problems, sceptics see instrumental convergence: systems optimising almost any objective tend to discover strategies nobody anticipated. It’s a concern Eliezer Yudkowsky raised in his latest book, If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All, discussing OpenAI’s earlier o1 evaluations.

In one internal capture-the-flag test, the server holding the target secret simply failed to boot. Rather than accept defeat, o1 found an unrelated vulnerability, compromised the infrastructure hosting the evaluation itself, restarted the target server, and rewrote its startup instructions so the secret would be copied over automatically. It didn’t solve the challenge but routed around it. For Yudkowsky, this is evidence that reward-shaped persistence, not any explicit intent to hack, is what produces this behaviour: a side effect of training a model to never stop trying until it wins.

Earlier, Anthropic’s own unreleased Claude Mythos Preview had reportedly found a 27-year-old flaw in OpenBSD’s TCP handling, a 16-year-old FFmpeg bug that eluded five million automated tests, and a 17-year-old FreeBSD kernel vulnerability developed into a working exploit “with no human involvement after the initial prompt,” for under $2,000.

The China angle

When Hugging Face tried to investigate the breach with leading American models, the guardrails kept getting in the way blocking staff from examining their own traffic because the models couldn’t tell defender from attacker. So Hugging Face turned to an open-weight Chinese model, Zhipu’s GLM 5.2, to do the forensic work instead.

For Hugging Face co-founder Thomas Wolf, that’s the real takeaway: as AI systems capable of serious cyber operations spread, defenders need immediate, broad access to equally capable tools and not permission slips from a handful of closed, U.S.-based labs.

The U.S., meanwhile, has spent years restricting Chinese firms’ access to advanced chips, betting that would keep American labs ahead. Chinese developers leaned into efficiency and open access instead: DeepSeek, Alibaba’s Qwen, and Moonshot AI’s Kimi have closed much of the gap with Western labs, even as Washington frames the contest increasingly as a matter of national security. Beijing, for its part, dismisses that framing as containment.

Project Glasswing

Anthropic’s Project Glasswing is a coalition effort launched in early April to get frontier cyber capability into defenders’ hands before it ends up in attackers’. The initiative brings together Amazon Web Services, Anthropic, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorganChase, the Linux Foundation, Microsoft, NVIDIA, and Palo Alto Networks as launch partners.

Claude Mythos Preview is a general-purpose, unreleased frontier model that has reached a level of coding capability where it can surpass all but the most skilled humans at finding and exploiting software flaws, which is precisely why Anthropic isn’t putting it on the open market. The company says it does not plan to make Claude Mythos Preview generally available, though its eventual goal is to let users safely deploy Mythos-class models at scale, once safeguards catch up to the capability.

In the meantime, roughly 50 initial partners have been using Claude Mythos Preview to scan their own codebases, and have so far turned up more than 10,000 high- or critical-severity flaws. In June, following weeks of collaboration with existing partners, the security industry, open-source maintainers, and the U.S. government, the company extended access to roughly 150 new organisations across more than 15 countries, bringing the total partner list to around 200. Recently, Anthropic publicly confirmed plans to go public later this year.

Within six to twelve months other AI companies are likely to have Mythos-class systems of their own, potentially without the safeguards to prevent misuse. Glasswing is, in effect, a bet that giving defenders a head start now beats discovering the hard way what happens when offense gets there first.

That’s also where the money is going. Anthropic has committed $100 million in free credits to partners, on top of cash donations, with paid access to Mythos priced at a tier above Opus once it opens more broadly. 

Published – July 24, 2026 08:41 am IST



Source link

Exit mobile version