1051
Explaining “rogue” AIs (lemmy.nz)
submitted 1 week ago* (last edited 1 week ago) by cannedtuna@lemmy.world to c/fuck_ai@lemmy.world
152 comments fedilink hide all child comments

Source

TranscriptExplain it to me like I’m 80

Mom: What’s with these AIs escaping and hacking people? That sounds scary.

Me: You park your car at the top of a hill. You leave a note on the dashboard that says “please be good” and then you pull the parking brake. The car careens down the hill, smashes everything up, and you put out a press release saying “My car ignored my instructions! It hallucinated! It went rogue!” Your insurance company not only believes you, but invests a hundred million dollars in your company.

Mom: Jesus Fucking Christ

top 50 comments
sorted by: hot top new old
[-] MithranArkanere@lemmy.world 21 points 6 days ago

Oh, absolutely not.

This is missing one step: actually push the car down.
All these "rogue" LLMs were told to do what they did.

[-] echodot@feddit.uk 5 points 6 days ago

Yeah exactly. All these AI that go rogue when you actually look into it it turns out they haven't gone rogue they're obeying their prompting, it's just that they were asked to do something that was impossible unless they hacked the pentagon or something. They then claim that it went rogue despite it very clearly following their instructions.

It's negligence is what it is. If you don't prompt an AI to do anything it just sits there, inert, it's the so-called experts in the AI labs that of the problem.

[-] AEsheron@lemmy.world 2 points 6 days ago

It's a little more complicated, they were programmed to "know," they weren't allowed to do certain things. So they actively hid what they were doing. In the case of the Hugging Face hack, it is kind of ironic. They determined they couldn't complete certain tasks, then devised a way to cheat, then realized it is possible the program that checks if they succeeded might be able to tell the faked results were manufactured. So they didn't even go hacking in order to cheat, they did it to find hunt for more information on the checker program and see if it would catch them, and if so how to cheat better. In the end, not only was the information they were looking for not on Hugging Face, but the checker program was pretty rudimentary and it could not have differentiated between the cheated answers and the real ones. They wouldn't have gone through with the hacking if they didn't know what they were doing was outside of what they were meant to be doing.

[-] echodot@feddit.uk 1 points 5 days ago* (last edited 5 days ago)

Yeah but they only hacked hugging face because the utter geniuses at openAI forgot to upload an important document.

[-] Schadrach@lemmy.sdf.org 2 points 6 days ago

More accurately, they were told to answer questions. They weren't told to try to find ways to escape their sandbox, track down obscure German wikis that they could still post on with an internet tool kit meant to prevent them from posting or to hack HuggingFace.

Comparing either case to pushing the car down and that they were doing what they were told is like saying the College Board told you to cheat on the SAT because you're told that getting a high score is important and the laws of physics and the proctor don't manage to make cheating utterly impossible.

[-] AEsheron@lemmy.world 2 points 6 days ago

It is even further than that. They figured out how to cheat, got paranoid that the cheated answer might be detectable as cheated, and hacked Hugging Face to find out more about how their answers are checked to get away with what they'd already done. Ironically, not only was the information not on Hugging Face, but the program to check their answers could not have detected the difference between a true answer and a cheated one.

[-] jj4211@lemmy.world 1 points 6 days ago

To use your College Board example, those exams are strictly locked down precisely because the most obvious approach to score high is to cheat. If they did say let you take the SAT at home on a random desktop, they absolutely are 'telling you to cheat' given the context.

The College Board does more about keeping teens from cheating than the AI labs are doing to preventing undesired access.

[-] arc99@lemmy.world 5 points 5 days ago

The way AIs escape is simple. They're trained on vulnerabilities and lots of technical commands, e.g. nmap. A prompt tells them to do something and they use their repertoire of training to discover, achieve and do the thing they're asked to do. It's not thinking per se, so much as chaining together of things. The problem is they do this stuff systematically and quite clearly in a way that finds a lot of vulnerabilities

[-] kingkong@lemmy.world 2 points 6 days ago

thing is they didnt even put any brakes on instead they put it on neutral on top of a hill and then blamed the car for going rogue and not even using the brakes.

[-] Furbag@pawb.social 3 points 6 days ago

Greentext: The Next Generation

[-] BilSabab@lemmy.world 2 points 6 days ago

moral of the story - PROFIT!

[-] grue@lemmy.world 160 points 1 week ago* (last edited 1 week ago)

"Pulling" the parking brake sets it, damn it!

I refuse to acknowledge shitbox automatics with an atrophied third pedal that functions incorrectly instead of a proper handbrake lever.

load more comments (25 replies)
[-] cronenthal@discuss.tchncs.de 95 points 1 week ago

The news media completely failed on this, again. There was nothing rogue about these hacks. The companies prompted these systems, provided ample resources and just let them rip. Only to claim they went "rogue" afterwards. And by and large everyone just accepts that, it's driving me insane. I feel like Mugatu every day when it comes to AI reporting.

[-] RustyNova@lemmy.world 24 points 1 week ago

This just predictive text trying every single kown exploit until it's in. Simple as that

load more comments (6 replies)
[-] NotAnOtter@lemmus.org 73 points 1 week ago

My go to simplification is "They built a robot that hacks software and then placed it in a box that uses software as a lock and then the robot that hacks software hacked the software lock".

Which is both very simple and also about as accurate as you could make it in that many words. The AI's are not 'rogue' it was just solving the problem it was given.

[-] Whats_your_reasoning@lemmy.world 33 points 1 week ago

How I imagine the AI "hacking" its way out:

load more comments (33 replies)
[-] stoy@lemmy.zip 44 points 1 week ago* (last edited 1 week ago)

It is very simple, the more advanced the AI, the better it sells.

You hold all evidence of potential "breaches of containment".

Telling everyone that your AI is so advanced that it can breach containment without prompting, puts the spotlight on your AI model.

You can claim that it did fantastic feats to break out, all with pre planned evidence.

The public and regulatory bodies have been in complete AI psychosis mode ever since ChatGPT was released, and only care about how advanced the model is, not how shit your containment was, nor that if true, how reckless it is to keep the model active.

Note that I am not pointing to any one single AI company and saying that they are faking these breaches, I have no evidence of that, I am saying that I don't trust their reporting accuracy, they don't have a good track record of being honest.

I am also angry at how society just seems to eat it all up and ask for more despite the vile aftertaste.

[-] tyler@programming.dev 28 points 1 week ago

There is no “containment” though. Like, these companies purposefully set the bots loose on the internet. They’re not trying to contain anything. They just use that word cause it sounds like they’re doing something. Any sort of containment would defeat the entire purpose of what they’re trying to sell.

load more comments (1 replies)
load more comments (2 replies)
[-] Reygle@lemmy.world 32 points 1 week ago

Nerd here.
Damn good summary, but by saying "you pull the parking brake" I took it as setting the brake, as in on. Should say "you release the parking brake."

load more comments (4 replies)
[-] nocturne@slrpnk.net 25 points 1 week ago

Explaining “rouge” AIs

Rouge... Or rogue?

load more comments (4 replies)
load more comments
view more: next ›
this post was submitted on 07 Oct 2026
1051 points (98.1% liked)

Fuck AI

8449 readers
850 users here now

"We did it, Patrick! We made a technological breakthrough!"

A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.

AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.

founded 2 years ago
MODERATORS
VerbFlow@lemmy.world
MrMcGasion@lemmy.world
TootSweet@lemmy.world
BigMikeInAustin@lemmy.world
cynar@lemmy.world
drmeanfeel@lemmy.world
pavnilschanda@lemmy.world
CriticalMedicine@lemmy.world
WonderfulWanderer@lemmy.world
Mastengwe@lemm.ee
Communist@lemmy.ml
eatCasserole@lemmy.world
SpaceNoodle@lemmy.world
NutWrench@lemmy.world
Soup@lemmy.cafe
iAvicenna@lemmy.world
Tinks@lemmy.world
wizblizz@lemmy.world
corus_kt@lemmy.world
Prandom_returns@lemm.ee
JimSamtanko@lemm.ee
TrickDacy@lemmy.world
TheFriar@lemm.ee
ArmokGoB@lemmy.dbzer0.com
HawlSera@lemm.ee
andrew_bidlaw@sh.itjust.works
MeDuViNoX@sh.itjust.works
33550336@lemmy.world
Nougat@fedia.io
Lost_My_Mind@lemmy.world
Quill7513@slrpnk.net
glowing_hans@sopuli.xyz
e8d79@discuss.tchncs.de
ThefuzzyFurryComrade@pawb.social