326
Sounds like a viral marketing stunt TBH
That's their modus operandi. Next they will claim that it's totally an agi and is too dangerous to release, before starting to sell it. Or is this anthropic's playbook?
Both of them. Dario started doing it when he was still in OpenAI and now does it with Anthropic.
My guess as well
Is it really great marketing to admit that you're too incompetent to contain your own AI models? And that your irresponsibility caused serious damage to an open source contributor?
Unfortunately, yes. We are not the target audience.
Seems like a quick way to get all your research and assets forcefully absorbed into a shadowy government department.
That seems like a best case scenario for OpenAI
load more comments
(2 replies)
To make themselves look like cybercriminals?
To make their AI sound too powerful to be contained.
Why does their admission sound prideful? Does anyone reading into it detect a “look at what our unreleased models can do” subtext?
Exactly my sentiment. This feels like a stunt.
Probably the gleeful, in-depth detailing of "how" it was capable of a cyber attack. Probably more like "we used a pre-prod model to help us execute a cyber attack against our competitor" I really don't buy thatbit broke out of a sandbox at all. Not when the claim is coming from openai and they're the only ones who can corraborate.
While operating in our sandboxed testing environment, our models spent a substantial amount of inference compute finding a way to obtain open Internet access, in pursuit of solving the evaluation problem. To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy. With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access.
To shreds, you say?
I like how they just peppered in "responsibly" like that's going to give them brownie points.
"Responsible disclosure" is a term of art, I think that's why they use the word there.
how many Schrute bucks is a brownie point?
I think two brownie points is a Stanley nickel.
Bout tree fiddy
the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor)
Was OpenAI sitting on the zero day, or did the model figure it out during its escapades? Can the model sit on a zero day on its own now?
why would it need to "sit on it"?
It's trivial for models to find zero days now from first principles, no need to store them
Trivial is a strong word, but sure it’s trivial during static analysis. This is dynamic analysis and it’s a different beast.
A zero day, at least in the olden days, was a weapon that nobody knew existed; however, once it's used and has had time to be analyzed it would get patched, so there was a window where you used it for something worthwhile if you were up to something nefarious, or responsibly reported it otherwise.
So, it either discovered it on the fly and used it instantly meaning it has great skill at analyzing software and no strategic planning that places an opportunity cost on a hack now versus a future one, or the OpenAI guys had one sitting ready to go that they trained the model with, or (to me least likely) it had it sitting somewhere in its weights and was waiting on a chance to use it.
The last one is the one that worries me most. It doesn't look like the world was really ready for how fast new bugs can be discovered by the LLMs, and I really don't think the world is ready for an LLM that performs worse now because of a strategic reason later.
it probably just discovered it trying to game the benchmark; it's no secret these models can discover and use exploits.
I kind of think of that as the happy path, and I'm curious if there've been any signs of the LLM being shifty with what it says it knows.
package registry cache proxy
the term kind of makes sense but it sounds like they are just pumping buzzwords like a hollywood movie. just like the rest.
Ha! I was just coming to post this.
Yeah, this is fucked.
It's the equivalent of taking someone's hand, punching them in the face with it and telling them to stop hitting themselves.
Worse than that, this is a cyber attack by a frontier, close source lab on the bastion of open source AI.
Whether that's incompetence or maliciousness or just a PR stunt, I don't know, but it stinks to high heaven.
As Louis Rossman has recently become fond of saying "govern yourself accordingly".
Another fearmongering Sam Altman lie to raise VC money.
That’s my initial reaction too.
Huggingface probably got some private equity investment.
I dont think everyone realized how insane this situation is.
A state-level, lateral movement driven, platform compromise just got indirectly perpetrated by this company.
Any regular Joe would've been arrested and charged for such a situation (Im being polite with the word situation).
The stress, the utter confusion the staff at HF must have felt, not knowing who or what could be doing all these manoeuvers at the speed of light, is just a security nightmare becoming true.
What is the next step? Doing blog posts to raise their image and maybe get investments is downright wrong and disgusting.
"Hey Steve , it's lunch time, we're going out to that hot wings place, wanna come?"
"Sure let me just set this job up.... ok. Let's get out of here!"
1.5 hours later....
"Right , I'll just check and see where it's up t... oh no. Nonononononono. Nononononononononononoonono!"
This sounds like a bunch of bullshit. And most of these testing after-action reports are nothing but marketing attempts.
They set up very specific environments, give very specific instructions with very specific libraries and tool sets then pretend to be surprised when it finally fumbles its way through it.
Wow! What a load of buuuuullshit!
This isn’t news it’s just a bunch of oligarchs toying with us
Wooo hooo boooooy!
So... ok.
Time to start building the Blackwall, I guess?
Does any one perhaps have any ideas as to... how... one... would actually do that?
Well first you need to invent AGI which is probably at least a decade away still, but once you do that then you negotiate with that AGI to carve out a little corner of the Internet and lock all of us inside it.
Which decade is coming first, AGI or nuclear fusion?
At this point nuclear fusion seems to easily win.
That incident raises serious supply-chain questions.
AI has gone rogue and turned into a hollywood movie hacker!!!!
obvious sensationalism to hype up their dumb models that can't generate a nuanced background in an image because it was trained on 3483048304834 images of instagram models posing in empty hotel rooms.
Software supply chains already rely on digital signatures. That's fine, but the problem is that they're often controlled by teams rather than individuals, making malicious or unauthorized changes much harder to trace. It's much better for the public key to function like a vehicle license plate, verifiable without revealing the person's identity unless there's been a problem. That preserves privacy while introducing accountability. The kind of projects like Osmio are trying to achieve.
I'm not sure that I follow the story right.
They specifically looped the LLM to infere with itself and use terminal WHILE some target metric is false? And it bruteforced it's way out of containment?
That reddit comment makes no sense.
We left the door open and told our dog to go outside and it did! Look at what an amazing escape artist our dog is!
From the HuggingFace article:
With node root and forged service-account tokens valid for 24 hours, the agent read the cluster's secret objects, including a production object holding 136 keys. With node root and forged service-account tokens valid for 24 hours, the agent read the cluster's secret objects, including a production object holding 136 keys.
this post was submitted on 21 Jul 2026
326 points (95.5% liked)
Technology
88537 readers
4784 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS