I'm no expert either, but...

People with wealth and power don't often invest trillions of dollars without a real prospect of a significant return on investment. There are obviously people with access to the best information available who believe that AI is going to deliver large scale disruption and concentration of wealth. I don't know the mechanisms they on the inside know, but their massive investment convinces me there are such.

If there is any truth to the stories that AI systems are able to find and exploit security vulnerabilities in systems then there is potential for exfiltrating sensitive, private information and exploiting that for financial gain and for directly manipulating those systems. Consider stock exchanges, commodity exchanges, logistics systems, SCADA systems, any systems that are the source or arbiter of truth for critical decision making. What large company or government doesn't have IT systems essential for management and control: accounting, payroll, ERP, CRM, CMS, data warehouses, reporting systems, etc. Much of our society is based on automated systems these days. While I think many stories are exaggerated, there are enough credible stories to suggest that AI systems are already and increasingly capable in this regard.

AI systems are designed to commit fraud very convincingly. They have already convinced many people to believe false information, about themselves and many other things. Gaining trust then using that trust to manipulate and defraud people has been used to manipulate markets since markets have existed. AI systems will facilitate and amplify what has always been. No need to speculate any new and ultra-sinister capability.

All sorts of media are increasingly overwhelmed by AI generated content and it is increasingly difficult to distinguish such from legitimate, truthful information. Misinformation campaigns are nothing new. The ability of them to manipulate society is well proven. AI is providing increasingly effective tools to conduct them, and AI systems are already pervasive in our media and communication systems.

AI systems are designed for and obviously effective at pattern recognition. There are patterns in markets and society generally. AI systems are increasingly powerful tools for finding and exploiting patterns in the markets and in society, including financial systems, political systems and other systems of power and influence.

And there is no need for 'rogue' AI systems intentionally and 'independently' operating contrary to the instructions of their morally upstanding operators. There are plenty of greedy sociopaths among those in control of them, perfectly willing to ruthlessly exploit anything for their own advantage, even if the AI systems are essentially passive.

On the other hand, swarms of agents are complex systems. Almost certainly chaotic. Almost certainly they will have unexpected behaviours: complex, chaotic systems are difficult to predict. Some of these might even be similar to prolonged, independent pursuit of objectives. Therefore, even if the intentions of the operators are ideally moral, legal and altruistic, there will be unintended consequences and some of those will be broadly harmful or beneficial to only a few. As AI systems gain control of more impactful systems, the impact of those unintended behaviours will increase. And there seems no end of people eager to give them access to the levers of power, just to see what happens. Even if they don't realize Skynet, there is significant potential for harm: intentional or otherwise.