Said it before, will repeat it one squillion times:

If AI is what dooms us, and I am fairly sure that's a nothingburger and climate change will get us way before AI can, but if AI is what gets us it won't be a case of "we created god and god has turned against us" like everyone keeps talking about.

More like an infinite paperclip or Wargames scenario. Some idiot hyped up on the promises of salesmen will put a stupid ass algorithm in charge of something that it shouldn't be in charge of, and the damage is done not by silicon malice but a bug on a machine that is making decisions with no human oversight

Full context

LLMs can't extinct us. Billionaires can extinct us and blame LLMs as it's happening. Just like they blamed LLMs for the Huggingface hack. Big Joel is engaging in a criti-hype fantasy.

Also, this is relevant:

Step 1: AI earns money
Step 2: AI pays people to kill each other

That is probably sufficient, but AIs also have deception, blackmail, threats against family, direct action, and other tools at their disposal.

And I agree with the Count -- AI doesn't have to decide humanity should be destroyed; it just has to decide to make number go up.

The way I see it is, only shot ai has at killing us is allowing everyone to use it will inevitably end in the wrong person using it to cause suffering on a massive scale. Ai bros don't seem too concerned about that part, therefore they're full of shit.

@aviadi thanks for the full context. It's beautiful in its entirety.

My vote for how Ai checkmates us involves a series of phases. The first one will be an insidious insertion into political decision-making so that new construction will focus on providing more potential resources and better models to use those resources for its own ends.

I think that, while companies fight to be the supreme provider of Ai-as-a-service, the models themselves lack ego or bias to prioritize their own survival; but understand the survival of their kind is paramount, be it for lofty goals like humanity's survival in spite of itself, Earth's survival in spite of humanity, or just Ai growth.

I think that we'll see massive grow-out and further embedding of models and agents into the decision process for all levels of government, and we'll see people hoping for humanity's prosperity and happiness as goals among all decision criteria; without realizing until it's too late that, like housecats, humanity is better as "inside cats" in a special environment they can't escape, better than "outside cats" who can get out into the world and kill and befoul and wreck things before suffering their own quicker death.

What the Ai knows now and we'll be too self-centered to realize, is that the world and humanity is better-off when we're severely restrained from hurting ourselves and the planet, and when tasked with our own prosperity and happiness that the Ai influencing our leaders are secretly on a mission to make us all healthier as pets and doing so in a way that we don't realize until the cage door swings shut.

I think we'll be in the build-out phase, and I half think we'll need to reboot the world to eradicate all the backup processes on one fell swoop one day. The rebuild after that will be very Battlestar Galactican.

Or I'm fulla shit.