1
RNZ’s take on AI today
(www.rnz.co.nz)
Funny how hf are not suing the pants off openai rn.
I have a suspicion this event was mutually beneficial propaganda, and question whether it happened at all.
Yep. Everything they've released has been disappointing as hell. AI can do some basic hacking, but anyone who can code can. It's not doing anything crazier than the most basic webapp hacks.
The only thing I find impressive is that AI can read a ton of code quickly and identify obvious issues in minutes, when it's usually days of work. It can lead to actually interesting stuff if you know what you're doing, but it is not at all close to being able to just hack stuff and spread on its own.
One thing they don't talk about much too is that with the Huggingface hack, they were telling the AI to hack stuff and complete some hacking tests. It was a hacking assessment. They got an AI to hack, it started thinking of shortcuts to solve the problem in interesting ways and hallucinated that Huggingface had data it needed, so it started trying to hack Huggingface and might have found something basic.
As someone in appsec, so the fuck what. You take any close look at a web app and you usually find stuff. AI is capable of that and it's not as impressive as it sounds. this is also a situation where they basically told the ai to hack shit and it did what they said... OH NO!
I am convinced whatever they did to sandbox it was weak as fuck on purpose and they were hoping for something like this, then just decided to let it go and see what happens. Or they were seriously fucking negligent and just told it to hack shit then ignored it for hours, which wouldn't surprise me.
If you have evidence to back up your suspicion, I suggest notifying the many reputable newspapers reporting the news and, as I now see, government legislators devoting their time to assessing the issue. They’d probably love to know that it’s bogus.
That is interesting. Perhaps it has to do with Nvidia’s acquisition of ~~OpenAI~~ (edit: oops) Hugging Face. Nvidia might not want to annoy a customer.
What I take from this is that it's an attempt to get China to agree to oversight, which I don't believe will happen (who would provide this oversight, the US?).
These incidents are unlikely to cause the extinction of human kind. The idea they might want to replicate is just applying a human thought process to what is definitely not intelligent. The biggest threat to humankind today is definitely the huge increase in carbon emissions that may well kill everyone before intelligent AI (or fusion) is even invented.
What I now think is plausible is something like a powerful AI being asked to solve global warming (let’s say), and deciding that massive reduction of the human population is the best way to achieve that, and then using its hacking tools to enact its own logic. Did I think that was possible before the Hugging Face attack? Nope, but I do now.
Don't get caught up in the AI terror hype. It's all about getting investors to dump more funds into businesses that are still in the red. They're just keeping the AI bubble going as long as possible.
These companies would likely need to close in days if not overnight if their investors pulled out immediately due to fear. This situation rhymes with the dotcom era. They all see a future with AI and a real technology, but they don't have a business model that's currently profitable using it, but they're expanding like they're ultra successful and pretend theyll get investor cash forever. It will run out.
The Huggingface hack is more the result of engineers asking AI to hack stuff, then being surprised when it hacked stuff. It wasn't just some weird AI deciding to hack things on its own. They were literally testing it in hacking.
They told it to hack stuff to win points, it decided to hack something else, it worked, and then they called it world news.
Not all hacks involve super crazy techniques. This isn't some super intelligence that can escape containment and spread itself. It's more than when you ask it to hack shit, sometimes it will, because not all hacks are that hard to pull off. If you can read and write code, you can pull off a lot of webapp hacks. I am not surprised, scared, or shocked at all.
This news is just more bullshit meant to make you think they've hit some new super intelligent level they haven't. LLM is a real and good technology that can be useful, but it's not a godlike super intelligence.
They just take advantage of people not understanding it's capabilities and hoping that investors throw more cash at them because they have a super weapon, or AGI, or self improving ASI that will change everything. the cash will eventually stop flowing and this news will fizzle out with, "... What the fuck was that AI craze of the 2020's? Why did we get tricked?"
I sure hope you’re right about fizzling out. I’m not so sure that it will as the cat is out of the bag, so to speak. What I’d really like to see is some good, robust regulation of AI and its applications, and a general recognition by the population that it’s worth taking seriously.
Totally correct and I’m getting wary of RNZ with all these hyperbolic scare mongering articles doing free marketing for these loss making grifters. OpenAI have proven they dont know how to sandbox a ‘swarm’ of autocompletion bots they themselves trained - though they continue to fail to understand their own creations since theyre probablistic. I call this a faulty shoddy product being pedalled hy a clutch of peychopaths being willfully negligent and just so happening yo have all the PR ready to go when it happens. And yet we parrot their own PR (marketing) news releases as fact. They are getting very desperate with their IPO looking further distant.
It’s important NZ does not end up in the same hole others will be in when the bubble bursts. A real shame RNZ a dont have real technical and levelheaded economic/business reporting to weigh-in. Pushback is needed to whats a ridiculous time of industrial and now social level mania to the tune of one trillion and counting (all debt aka circular financing) which will never be recouped let alone turned into any kind of productivity short of a bit of coding assitance.
NZ should be independent on this. But here we are yet again kowtowing to the big seven plus these new hucksters. What an embarrasment.
I'm skeptical that AI will kill all humans, terminator style, but it seems quite credible that some combination of the AI investment bubble and AI being used to intervene in or manipulate markets will cause so much economic disruption that many people will die. Possibly through global economic collapse - an extremely severe recession. Possibly through extreme concentration of wealth - because the current concentration of wealth is not yet extreme enough for the would be multi-trillionaires. Either way, leaving a large part of the population with essentially nothing.
I’m interested to know by what mechanisms you think that AI could manipulate markets. Not my area of expertise.
And yes — I generally agree with the thrust of your post about the concentration of wealth and wonder what the billionaires’ end game is. History teaches us that desperate people revolt, and revolutions are not pretty.
I'm no expert either, but...
People with wealth and power don't often invest trillions of dollars without a real prospect of a significant return on investment. There are obviously people with access to the best information available who believe that AI is going to deliver large scale disruption and concentration of wealth. I don't know the mechanisms they on the inside know, but their massive investment convinces me there are such.
If there is any truth to the stories that AI systems are able to find and exploit security vulnerabilities in systems then there is potential for exfiltrating sensitive, private information and exploiting that for financial gain and for directly manipulating those systems. Consider stock exchanges, commodity exchanges, logistics systems, SCADA systems, any systems that are the source or arbiter of truth for critical decision making. What large company or government doesn't have IT systems essential for management and control: accounting, payroll, ERP, CRM, CMS, data warehouses, reporting systems, etc. Much of our society is based on automated systems these days. While I think many stories are exaggerated, there are enough credible stories to suggest that AI systems are already and increasingly capable in this regard.
AI systems are designed to commit fraud very convincingly. They have already convinced many people to believe false information, about themselves and many other things. Gaining trust then using that trust to manipulate and defraud people has been used to manipulate markets since markets have existed. AI systems will facilitate and amplify what has always been. No need to speculate any new and ultra-sinister capability.
All sorts of media are increasingly overwhelmed by AI generated content and it is increasingly difficult to distinguish such from legitimate, truthful information. Misinformation campaigns are nothing new. The ability of them to manipulate society is well proven. AI is providing increasingly effective tools to conduct them, and AI systems are already pervasive in our media and communication systems.
AI systems are designed for and obviously effective at pattern recognition. There are patterns in markets and society generally. AI systems are increasingly powerful tools for finding and exploiting patterns in the markets and in society, including financial systems, political systems and other systems of power and influence.
And there is no need for 'rogue' AI systems intentionally and 'independently' operating contrary to the instructions of their morally upstanding operators. There are plenty of greedy sociopaths among those in control of them, perfectly willing to ruthlessly exploit anything for their own advantage, even if the AI systems are essentially passive.
On the other hand, swarms of agents are complex systems. Almost certainly chaotic. Almost certainly they will have unexpected behaviours: complex, chaotic systems are difficult to predict. Some of these might even be similar to prolonged, independent pursuit of objectives. Therefore, even if the intentions of the operators are ideally moral, legal and altruistic, there will be unintended consequences and some of those will be broadly harmful or beneficial to only a few. As AI systems gain control of more impactful systems, the impact of those unintended behaviours will increase. And there seems no end of people eager to give them access to the levers of power, just to see what happens. Even if they don't realize Skynet, there is significant potential for harm: intentional or otherwise.
this post was submitted on 14 Sep 2026
1 points (60.0% liked)
Aotearoa / New Zealand
2243 readers
4 users here now
Kia ora and welcome to !newzealand, a place to share and discuss anything about Aotearoa in general
- For politics , please use !politics@lemmy.nz
- Shitposts, circlejerks, memes, and non-NZ topics belong in !offtopic@lemmy.nz
- If you need help using Lemmy.nz, go to !support@lemmy.nz
- NZ regional and special interest communities
Rules:
FAQ ~ NZ Community List ~ Join Matrix chatroom ~ Alternate frontends ~ Donate
Banner image by Bernard Spragg
Got an idea for next month's banner?
founded 3 years ago
MODERATORS