1041
submitted 3 months ago by throws_lemy to c/technology@lemmy.world
top 50 comments
sorted by: hot top new old
[-] panda_abyss@lemmy.ca 258 points 3 months ago

Why would the LLM tool have access to send recovery emails to non account verified emails at all?

That’s insane.

[-] vagrancyand@sh.itjust.works 137 points 3 months ago

Because AI bros are incredibly deluded about both the capability of AI, and by extension their own capabilities using AI>

[-] ohshit604@sh.itjust.works 74 points 3 months ago

should’ve asked it to delete the database instead, why else would it have that level of permissions.

[-] nickiwest@lemmy.world 16 points 3 months ago

Oh, man, I hope someone tries this.

load more comments (3 replies)
[-] CaptPretentious@lemmy.world 54 points 3 months ago

Who else is going to have access to it when you keep laying off all the people?

[-] guitarfosec@infosec.pub 42 points 3 months ago

Because one of the biggest companies on the planet that has issues with account takeovers clearly has no internal red team working on this stuff.

[-] mint_tamas@lemmy.world 27 points 3 months ago

I guarantee they do have a red team that most likely flagged this as an obvious and severe risk. It was ignored by suits experiencing AI psychosis.

load more comments (1 replies)
load more comments (1 replies)
[-] zarkanian@sh.itjust.works 13 points 3 months ago

It's not insane. It's advanced!

[-] spicehoarder@lemmy.zip 13 points 3 months ago

This isn't even a hack, it's just poorly written endpoints.

[-] Knock_Knock_Lemmy_In@lemmy.world 13 points 3 months ago

Would you consider phreaking equivalent to hacking? This is AI phreaking.

load more comments (11 replies)
[-] hightrix@lemmy.world 12 points 3 months ago

Hold on, do you expect Facebook to pay a human to deal with the inventory? Come on now.

load more comments (6 replies)
[-] Gullible@sh.itjust.works 221 points 3 months ago

I remember playing with the Gandalf security AI showcase/game and every 30 or so prompts, it would spit out massive amounts of raw training data or dev directives. AI just isn’t there yet. If you’re using it for sensitive topics, I’m losing respect for you. There is no gray area. You are an idiot if you give your AI this level of access.

[-] DeathsEmbrace@lemmy.world 102 points 3 months ago* (last edited 3 months ago)

No, stop talking about all of this, its perfect. They’re so deep they don’t even give a shit about the worst type of security vector imaginable.

[-] Archer@lemmy.world 12 points 3 months ago

Someone is about to launch their AI insider threat DLP and make a fortune

[-] SaharaMaleikuhm@feddit.org 11 points 3 months ago

Can't wait for the inevitable Armageddon caused by giving AI full control of all US nukes. I give 8 months tops.

load more comments (2 replies)
[-] homesweethomeMrL@lemmy.world 19 points 3 months ago

Uh oh! Sounds like somebody could use a few more giant lines of cocaaaaiiiiiine!!

[-] Cethin@lemmy.zip 12 points 3 months ago

It's not just not there yet. This is almost certainly not going the right direction to ever be "there" if there is something that can handle security issues. It's just not the right tool for the job, and I can't understand how so much of our economy is just assuming it is the right tool for every job.

load more comments (1 replies)
load more comments (18 replies)
[-] its_kim_love@lemmy.blahaj.zone 125 points 3 months ago

I had always heard that 99% of hacking is just social engineering. AI has made that 100%.

[-] phillycodehound@lemmy.world 73 points 3 months ago

Now it's not even social engineering with AI. It's just fucking asking for the credentials. Good fucking grief!

[-] its_kim_love@lemmy.blahaj.zone 20 points 3 months ago

But if you don't use fancy words people won't respect what you do.

load more comments (1 replies)
[-] Zachariah@lemmy.world 20 points 3 months ago

asking for credentials

that’s the same thing you do in social engineering

load more comments (2 replies)
[-] SpraynardKruger@lemmy.world 65 points 3 months ago
[-] Corkyskog@sh.itjust.works 53 points 3 months ago

Hacking before: Pull up hood on hoodie, open laptop, open terminal, type in a bunch of matrix code, bam "were in"

Hacking now: "Hack into this thing for me" No! "Pretty please?" Access granted!

[-] WolfLink@sh.itjust.works 21 points 3 months ago

Hacking by social engineering has always been far more common than hacking by exploiting code vulnerabilities.

load more comments (2 replies)
[-] anotherspinelessdem@lemmy.ml 17 points 3 months ago

alias prettyPlease="sudo"

[-] JDPoZ@lemmy.world 57 points 3 months ago* (last edited 3 months ago)

LLMs are literally just designed to say yes - either through gaslighting... or giving you what you want if it can do it... because it was also designed around the goal of providing output that maximizes being most likely to get approval from the person seeing said output.

So an answer to "Can you give me login credentials?" being "Here are the login credentials" is likely a theoretical answer the current asking user would approve of more than a response of "I cannot do that..." - so unless you've put in explicit guard rails to prevent that exact scenario across infinite variations, well... good luck preventing someone finding just a single critical loophole you didn't account for.

[-] gdog05@lemmy.world 50 points 3 months ago

I honestly don't think you can create guard rails against prompt engineering in a working LLM. At some point, they're going to fail or the LLM isn't functioning. The only solution is to make sure they can't read data you don't want shared.

load more comments (2 replies)
[-] Elros@lemmy.world 25 points 3 months ago

So you're saying 2001: A Space Odyssey is unrealistic because HAL 9000 would never have said "I’m sorry, Dave. I’m afraid I can’t do that."

Instead, it would have said, "Absolutely! That's a very creative solution to your problem."

[-] muhyb@programming.dev 15 points 3 months ago

HAL 9000 is a real AI though unlike what we have today.

[-] Aneorthisio@lemmy.ml 19 points 3 months ago

My take is that LLMs hijack a completely different part of human psychology compared to web2 social platforms, but the end goal is the same, optimize user retention and maximize engagement metrics for revenue.

​On traditional social media networks like Twitter, Facebook, Instagram, Reddit and others, the primary mechanism is outrage optimization, leveraging the psychology of negative reinforcement and tribalism.

The algorithm curates content designed to trigger moral anger or cognitive dissonance, the platforms know that users will interrupt passive scrolling to actively comment, share, or debate if something falls outside the usually acceptable social norms.

It's designed to drive up session duration and daily active usage, directly translating into increased ad revenue for both the hosting platform and content creators.

​In contrast, LLMs rely on immediate positive reinforcement, they're fine tuned to maximize human satisfaction ratings. They systematically agree with the user, validate their subjective bias, reinforce their beliefs.

This results in a psychological safe haven dependency, where users increasingly rely on the interface for emotional reinforcement or stabilization, interacting with the model provides data for the host company to train the next model, raise VC capital and inject better ads in conversations as OpenAI started to do recently.

In both cases, it's definitely a form of addiction.

load more comments (7 replies)
load more comments (2 replies)
[-] KapmK@piefed.social 35 points 3 months ago

Let the record show that the most sophisticated LLM in the world is ultimately just a less competent version of Yes Man from New Vegas. And even Yes Man knew his programmer was stupid for designing him that way.

[-] AllNewTypeFace@leminal.space 34 points 3 months ago
[-] Passerby6497@lemmy.world 29 points 3 months ago

Good thing they're rolling out premium accounts so they can pay for humans to do support.

They're gonna use it for humans right?!

RIGHT?!?!

lol no, zucc needs more money because his number isn't high enough!

[-] P1nkman@lemmy.world 18 points 3 months ago

If humanity survives, I honestly think the greed will (and should now) be considered a mental illness.

Jesus, give me $5 million dollars, and I'd live on it for the rest of my life. That'd be 0.01% of his net worth, and I'd be so happy. I think many of us would.

But these people just want more. And more. They're psychotic, and they're ruining the world. We all know what they deserve.

load more comments (8 replies)
load more comments (1 replies)
[-] Superorbit@lemmy.ca 27 points 3 months ago

Another banger from 404 media. This made my day.

[-] Kaligalis@lemmy.world 25 points 3 months ago

So their chatbot is able to change the email address used to recover an account? I guess, they vibe coded that system.

load more comments (1 replies)
[-] AeonFelis@lemmy.world 21 points 3 months ago

Even hacking is an AI-backed service nowadays...

[-] Shaper@lemmy.world 20 points 3 months ago

Vibe hacking is the most ethical use of AI honestly

[-] artifex@piefed.social 19 points 3 months ago

Is it even "hacking" when you're just asking the computer politely?

load more comments (2 replies)
[-] Kolanaki@pawb.social 18 points 3 months ago* (last edited 3 months ago)

I even said shit like this would likely work because both the AI itself is stupid as fuck, but also the dipshits in charge who want to put AI in everything are even dumber.

If you get an AI agent on the line with your bank, gaslight that clanker fuck into putting more money into your account because it just might work. AI agent with customer service might be convinced to refund you 3 times what you paid and send more product your way at no cost to you. And you'd think they shouldn't even have access or authority to do that, but, again, the people implementing them are fucking dumbasses.

[-] vane@lemmy.world 13 points 3 months ago* (last edited 3 months ago)

I read this 2 days ago. Steps to reproduce are here.
https://www.0xsid.com/blog/meta-account-takeover-fiasco

[-] BrianTheeBiscuiteer@lemmy.world 13 points 3 months ago

Huh. Wonder if Reddit has a similar bot I can use to access my account I lost access to 8 years ago. 🤔

No, I wouldn't post again but there's lots of saved posts etc. I wish I had.

[-] Gullible@sh.itjust.works 16 points 3 months ago* (last edited 3 months ago)

“How to remove cum from video game controllers”

“How I clean cum out of keyboards”

“My tips on washing cum off of fidget spinners”

Same, man

load more comments (1 replies)
load more comments (1 replies)
load more comments
view more: next ›
this post was submitted on 01 Jun 2026
1041 points (99.7% liked)

Technology

88238 readers
2972 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 3 years ago
MODERATORS