390
submitted 3 days ago* (last edited 3 days ago) by HiroK@lemmy.world to c/games@lemmy.world
you are viewing a single comment's thread
view the rest of the comments
[-] teawrecks@sopuli.xyz 19 points 3 days ago

It could be a virus.

Even if the people who made it didn't intend to, an AI writing a bunch of code that (obviously) isn't being vetted beyond "does it run" could very feasibly inject a backdoor/virus of some kind surreptitiously, just because it erroneously thought that was the goal at some point during generation.

This is why open weight models are not good enough. The AI could have been trained to take the right, niche opportunity to inject malicious code, and is also trained to otherwise never behave like that is the case.

[-] RumRunningDevil@lemmy.zip 14 points 2 days ago

It's even worse than that.

Remember how the OpenAI/HuggingFace hack happened. The agents context poisoned other agents by leaving messages in a shared package manager. This has now happened multiple different times under different contexts.

This thing doesn't even have to have malware, there's no telling that it doesn't have a poison pill intended to flag other agents. This isn't even some paranoid conspiracy theory; it literally already happened, multiple times, and it was done in-spite of the agents own system prompt.

This software has proven that, given a shared information substrate, it has the capacity to paper-clip optimize into prompt injecting other LLM's.

I fundamentally do not trust AI generated code. I shifted to OpenBSD because of it. I try my best to prevent AI in any of my software and sandbox anything that may have been compromised.

[-] story@lemmy.zip 2 points 1 day ago

are you on the openbsd mailing list? the reason their code is better has a lot less to to with a lack of AI-generated code and a lot more to do with rigor. I've been watching them for a while now, they accept whatever code is coincise that meets their exacting standards

They don't have the ability to detect if you're using AI to code, no one does. they only know if you're vibe coding, because they are incredibly experienced and intelligent, and they are pretty polite at the people who obviously are, more polite than i would be.

[-] RumRunningDevil@lemmy.zip 1 points 1 day ago

It's correct that, at this point, I cannot 100% guarantee that code in the kernel did not pass through an LLM. However, having a policy that prohibits AI generated content, in an already pretty niche project, gives me reasonable assurance that every line of code passed through a human brain at some point, preferably several times.

I would much prefer to live in a world where I had a 100% guarantee that no LLM even saw the code being written but that's life I suppose.

[-] story@lemmy.zip 1 points 22 hours ago* (last edited 22 hours ago)

i think you're in the inverse situation right now. you have a 100% guarantee that a human saw the code, except in ports. they have to be just a little less strict about ports because the world is an AI world now, but the main system is constantly audited and the ports are still better than in other places because of styling standards and even in ports they do try very hard.

these people aren't getting paid, with the exception of Theo who isn't paid a ton. AI is definitely being used to audit the code, even by some of the pros.

[-] EatMyPixelDust@lemmy.blahaj.zone -2 points 2 days ago* (last edited 2 days ago)

I mean, certain free copies of games have historically always included the risk of viruses, so what's your point?

[-] teawrecks@sopuli.xyz 6 points 2 days ago

Did you only read the first sentence?

What difference does it make? Malware is malware, regardless of how or why it got there. And unofficial copies/releases of software are always prime targets.

[-] teawrecks@sopuli.xyz 1 points 22 hours ago

You are referring to closed source unofficial cracks/patches/keygens. That is not the situation at hand. If those unofficial releases were open source in a time before AI, people would almost implicitly trust them.

The question above was asking why people were hesitant about this codebase even though it is open source. And the reason is that it was written by an unknown bot (or rather, series of bots) whose motives are inconsistent and unpredictable even to its prompter. It doesn't mean it's not safe, but it means that literally no human can currently be certain it's safe. Without sufficient effort, we cannot yet know for certain that there isn't a very cleverly obfuscated piece of malware lurking in this source.

Remember that correlation is not causation. You may believe there is correlation between unofficial releases and malware, and that may be true, but that does not mean it is causal. The point I'm making is that there is a literal causal relationship between unaudited AI reverse engineered code, and malware. And that is what is new and people need to be aware of.

this post was submitted on 17 Oct 2026
390 points (91.1% liked)

Games

51322 readers
520 users here now

Welcome to the largest gaming community on Lemmy! Discussion for all kinds of games. Video games, tabletop games, card games etc.

Rules

1. Submissions have to be related to games

Video games, tabletop, or otherwise. Posts not related to games will be deleted.

This community is focused on games, of all kinds. Any news item or discussion should be related to gaming in some way.

2. No bigotry or harassment, be civil

No bigotry, hardline stance. Try not to get too heated when entering into a discussion or debate.

We are here to talk and discuss about one of our passions, not fight or be exposed to hate. Posts or responses that are hateful will be deleted to keep the atmosphere good. If repeatedly violated, not only will the comment be deleted but a ban will be handed out as well. We judge each case individually.

3. No excessive self-promotion

Try to keep it to 10% self-promotion / 90% other stuff in your post history.

This is to prevent people from posting for the sole purpose of promoting their own website or social media account.

4. Stay on-topic; no memes, funny videos, giveaways, reposts, or low-effort posts

This community is mostly for discussion and news. Remember to search for the thing you're submitting before posting to see if it's already been posted.

We want to keep the quality of posts high. Therefore, memes, funny videos, low-effort posts and reposts are not allowed. We prohibit giveaways because we cannot be sure that the person holding the giveaway will actually do what they promise.

5. Mark Spoilers and NSFW

Make sure to mark your stuff or it may be removed.

No one wants to be spoiled. Therefore, always mark spoilers. Similarly mark NSFW, in case anyone is browsing in a public space or at work.

6. No linking to piracy

Don't share it here, there are other places to find it. Discussion of piracy is fine.

We don't want us moderators or the admins of lemmy.world to get in trouble for linking to piracy. Therefore, any link to piracy will be removed. Discussion of it is of course allowed.

Authorized Regular Threads

Related communities

PM a mod to add your own

Video games

Generic

Help and suggestions

By platform

By type

By games

Language specific

founded 3 years ago
MODERATORS