275
I always wonder what happened to those 500 people who signed a letter saying they'd quit OpenAI if Sam Altman wasn't brought back
the fact that they didnt cut the internet off open ai and then complain that open ai got to the internet is mid.
How about somebody prosecutes the criminal acts of these companies instead?
(Disclaimer: I'm just memeing and reserve the right to change my position if I learn of bad unintended consequences.)
This is stupid. Open-weight models that can largely do what the OpenAI/Anthropic models did are already out in the wild.
Do these people not understand that you can't put the cat back in the bag?
It’s not at all stupid, because there’s no such thing as a “rogue agent,” only rogue users of AI. With proper airgapping, there’s no such thing as a “rogue agent.”
But isn’t that the problem? You can’t force a rogue user to do anything; or they wouldn’t be rogue. So why should they apply air gapping to their agents?
So why should they apply air gapping to their agents?
Because if they don't, they could face jail time personally or their company is fined to death? That's how you apply corrective pressure to those thinking about "going rogue".
I must be missing something, because you're getting a bunch of upvotes. I don't disagree with anything you said, but it feels like a total non-sequitur to what I was saying.
Yes - proper air-gapping is a good idea. But asking the big AI labs to air gap is kind of pointless. Liability already gives them legal incentives to avoid incidents. If that didn't stop them, a letter signed by a bunch of academics sure won't. More importantly, open-weight models capable of doing all the things that we're concerned about are already in the hands of bad actors around the world.
This letter is asking us to lock the front door of the house when the back wall is missing.
The majority of lemmy seems to not understand the concept of a pandoras box and tends to just talk in circles are the reality of the situation to various degrees.
People are too hung up on hating AI to actually stop and look around them and realize what is physically and technically possible basically doesn't align at all which 99.99% of what people want to happen.
Stricter regulation and holding people responsible for their agents should be a thing yes and no one sane will disagree.
But the reality of how that's even do able is VASTLY different then what people want it to mean.
No it isn't. We could have comprehensive regulations by this time tomorrow if the people in power had the will to.
Liability already gives them legal incentives to avoid incidents.
That's quite optimistic, especially since there's almost no enforcement of liability.
... open-weight models capable of doing all the things that we’re concerned about are already in the hands of bad actors around the world.
Computational resources are the limiting factor. AI companies have resources which no independent actor could ever dream of, which makes them far more dangerous to the general public. A chihuahua with an attitude problem can only do so much damage. A Great Dane with an attitude problem is a threat no one can afford to ignore.
You think scammers in Africa/India are capable of running a local open weight model equivalent to the performance of a cloud frontier model?
You don't need something that power to do damage. And fuck the frontier model of 6 months ago and now are basically two entirely different things. In another six months?
Claude fable had a bunch of concerns on release and you don't think what evers out in the next year isnt going to beat it?
Getting a bunch of beat up Mac minis together and running something isn't that hard for a mid tier operation. They don't have to be fast. They just have to turn on.
The ones in Russia definitely are. They don't have to be local models - you run them in the cloud like anyone else.
The idea that the frontier labs have LLMs with special capabilities that make them dramatically better than the top open-weight models is largely marketing.
The reality is that most systems are poorly secured, and LLMs make exploiting vulnerabilities fast, cheap, and easy. What would have taken a trained security researcher weeks before can be done by a much less skilled attacker with an LLM in a few hours.
This is correct. We can thrash all we want about slop and hallucinations, but from a defensive posture position, the models only need to be right some of the time and they still constitute a threat.
Knowing it sucks and pretending it doesn’t exist are two different things.
the models only need to be right some of the time and they still constitute a threat.
Right!?!
And with how cheap and fast they are to run, they don't have to be right that often as long as a separate run can verify/validate claims. In a lot of ways what the LLMs do should probably be thought of as just a very sophisticated sort of brute force approach. You're effectively just searching the space of combinations of known vulnerabilities. It doesn't require particular cleverness, just an encyclopedic knowledge, a reasonable way to prune the search space, and the ability to put in the compute time.
I think this is suggesting that all AI models have restrictions. Yes, some people will ignore those restrictions, but at least there will be some recourse as there will be actual rules put in place. If a company or an individual does something reckless with AI that affects another company or individual negatively the offender may be able to currently say "it wasn't me it was the AI/I didn't know". They may not have technically broken any rules since the current law is not able to account for AI agents acting somewhat autonomously. If there are rules in place then at least we can point to that and say "it doesn't matter that you did not explicitly instruct it to do X, you should have had the following things in place to prevent that".
The fact that some people may be able to run local models and may never be caught does not mean that we shouldn't have the infrastructure in place to stop the harm where and when we can.
Technology is famously hard for the law to keep up with, but I don't think that's what's going on here. We have laws for when a person uses a technical tool to obtain unauthorized access, exfiltrate data, or do other malicious things. These don't necessarily require any specific intent. If you've set up an AI tool and pointed it in the direction of hacking without proper airgapping, you've probably violated the CFAA if you're in the US when it's successful. Other countries generally have similar laws. Prosecutors don't need to try someone if it's really ambiguous or unclear what they actually did. Is this really unreasonable under the legal system?
To the extent there's a gap, it's the system around the law. Lawyers and judges aren't technical experts, and may not recognize "it wasn't me, it was the ai" as the same argument as "it wasn't me who shot him, it was my gun", in that it's usually foolish and unserious, but there can be a convoluted scenario in which it's a good argument.
this post was submitted on 15 Oct 2026
275 points (98.2% liked)
Technology
88796 readers
4252 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS