AI coding tools are pretty good at finding bugs and make it easier and faster to fix them.
Importantly, security vulnerabilities in new releases seem to stay roughly the same and the change to which bugs count as CVEs was around version 6.8 already.
do we have one for regressions ? because that seems to be exploding for every other project.
The difference is that Linus uses LLM's as a tool, not as a hands off content creator.
The LLM points out something a human missed in the same way spell check finds typos. Then a human reads the suggestion and writes the fix.
if that were entirely true, what i said would not be.
but yes, in the hands of a skilled professional with over 30 years of experience on a single project it can probably be useful.
Your statement was about other projects having regressions because developers are using AI to write code for them completely hands off and compared it to Linux where AI is used to make suggestions but a human is ultimately responsible and has to write the patch.
Apples and Oranges.
i made no claims about their methodology.
"if that were entirely true, what i said would not be."
I explained the difference in methodology and you doubled down as if they were the same.
in that case i should have been more clear. i do not believe it's strictly one or the other, which is why the word "entirely" is in there. you're generalising over a huge range of developer quality and basically saying that they are all worse at building software than linus, despite the fact that we have been seeing regressions in projects like rsync, which has one of the most disciplined maintainers in the foss ecosystem.
you're generalising over a huge range of developer quality and basically saying that they are all worse at building software than linus
You claimed that Linux could see regressions just like all other projects DESPITE the different methodology.
basically saying that they are all worse at building software than linus
That is claim that any AI use of any type results in regressions. That is obviously false.
Just because AI suggested using glue on pizza doesn't mean that everyone now puts glue on their pizza.
i claimed no such thing. i asked for a graph because i was interested to know if that was the case.
and what we've been seeing in the industry lately is that llm use of any kind results in more regressions, because more code is created and it tends towards average quality.
Those other projects are not at high quality control like Linux. They probably vibe slop it and introduce a lot of regressions. Such data would be interesting on Linux too, just to have some comparison. But I don't think it's exploding regressions, as there is no indication for that so far.
regressions ? because that seems to be exploding for every other project.
Would those be regressions with, or without automated regression tests? (Assuming without.)
When you've got a minion at your disposal who can code really really fast, one good use for it is to have it code automated unit and regression tests.
For anyone curious about the number of bugs introduced in each kernel release, LWN.net has a plot of Bugs fixed and introduced in each kernel release.
However, it has the obvious limitation that recent releases will appear to have fewer commits introducing bugs, simply because kernel developers haven't had as much time to find and fix the bugs introduced in those releases. So it'll be a while before we can actually see the consequences (positive or negative) of more widespread LLM usage by kernel developers
As somebody who is quite critical of AI, I will say, it is a fantastic tool for finding bugs early. I advise getting all your changes reviewed by a decent LLM before you push them. It often finds edge cases I forgot in my testing
To add to this, allium is a weirdly effective tool for finding bugs and clarifying intent despite being nothing more than a prompt that you present to a frontier model.
I’ve started using allium spec files as a replacement for handoff documents. It records what you want your program to do without any of the implementation details that might pollute an LLM’s context window.
That's a pretty big limitation! They shouldnt have bothered plotting that, it's basically just a plot of time.
It will be fascinating when the data is available, to see a graph showing the number of security vulnerabilities added to the kernel with each commit, obviously they'll have to be found and patched before that can be determined.
I haven't even looked at this yet but someone posted https://lwn.net/Articles/914632/ elsewhere in the thread, so sending it to you. As noted, less time to find in recent versions.
ai, protecting people from ai.
yes, you need to find the vulnerabilities before the hackers and it just so happens that LLMs aren't terrible at it.
I think dusk means it would also be difficult for hackers to find these bugs without AI, which probably is true (given how long some of them seem to have existed). Hard to say how it balances out in the end though.
Some people think its bad that so many CVEs are fixed. Because they think without Ai we wouldn't have so many CVEs in the first place... go figure (not my opinion, but I saw such type of comments).
It's natural to think that way if you don't think the implications through - like the infamous WWII plane showing holes in the fusilage and the implications on which parts of the aircraft needed reinforcement. :)
AI coding tools are pretty good at ~~finding~~ writing bugs and make it easier and faster to ~~fix~~ write them.
If you checked the commits you would see that some of the bugs in Linux being found with AI tooling have been present for many years.
I encourage everyone to go have a read for themselves.
Linux hasn’t become some slop treadmill like you’re implying.
Been away for almost a year and came back to many here thinking they're smarter than Torvalds. It's stunning. Goes with what I've been saying about broad DC hate, it's going to cause friendly fire.
If you put someone on a pedestal, they will eventually shit on you.
Such a lemmy moment, unable to admit AI is capable of anything that might be considered useful.
Yes AI is good at writing bugs, it's also good at finding exploits.
"such a lemmy moment" 🤣🥱
Some of it is due to AI but much of it is people finally paying attention to security. For a long time no one thought about security vulnerabilities. These days many bugs in Linux are considered a security issue and thus gets there own CVE
This was never an issue before, so it just appears as tho it's adding bugs to be fixed later. Unless all these updates are strategically going somewhere.
Many of the bugs have been present for years. This is about finding bugs that humans didn't, then humans fixing the bugs.
These were not a problem, because no one found them, but they were still there. Now they are being found, if not by the folks patching them, then by those looking to exploit them. There is no way to navigate this than to patch whatever is found.
Many governments and secret groups sit on Zero day exploits for years until they need to use them.
Average people are finding them easier now using AI, which is a good thing. Instead of 3 letter organizations sitting on them.
A community for everything relating to the GNU/Linux operating system (except the memes!)
Also, check out:
Original icon base courtesy of lewing@isc.tamu.edu and The GIMP
