Ive been saying this. The only way this succeeds is to sell this to businesses, but even then I feel like companies are realizing it's way to much money to actually justify it because they want to reduce workforce and save money but this will not do that. In the end they get a great deal on the first few years before the contract is up and then have to renew at way higher pricing. Not to mention they have less control over the company and will be paying yearly maintenance fees that probably cost more than the raises they wouldnt have given their employees but now have to pay up those crazy fees every year no matter what. Yeah it's absolutely going to fail for most things for the forseeable future. Someone will find a way to make it work eventually, but its still a ways off.

And Tesla is still around, so…

Now that people are shunting easier requests to local models and sending the harder problems out, the hyper scalers have less income, so they have to charge more.

While you're 100% correct about the fact that this is happening, and it is a serious problem for the hyper-scalers, it's really, really important to note that even before this started happening, LLMs were already hilariously unprofitable, both on the software and the hardware side. For the AI companies, there is simply no way to charge people what the models actually cost to run and still turn a profit. On the data-centre side, the turnover on burned out GPUs is going to annihilate any hope of profitability. They're treating as capex what is essentially a consumable and their business models simply do not account for this reality.

The behaviour you're identifying here is basically just a swift kick in the guts to someone who was already bleeding to death from several bullet wounds. Seriously unhelpful, but not remotely the impending cause of death.

If you watch your PC grind to a halt generating 1 image over 5 minutes, it makes it less appealing to queue up 1,000 and cherry-pick the best; let alone making videos. It's too easy right now to use a ruinous amount of compute power for something stupid like a fake dog video without even realising the cost

With local you're limited in performance by the hardware you can buy, which is a huge up-front that most people would balk at. Hosted will give you as much performance as you could ever want billed by usage with several profit vectors built-in, but the customer sees smaller dollar number and better apparent performance and thinks they're getting a deal. Car dealerships do the same thing with selling the monthly payment and not the actual sticker price.

The environmental difference then could be significant. Users taking the hosted route can choose to use much more resource-intensive models than a local PC could. They might see a $300 AI bill as a little pricey when it is usually $100, but figure that's cheaper than buying a PC that could produce the same result, just at maybe half speed. This would result in a much higher energy cost from the hosted solution compared to the PC.

Electricity burned is the same. But so is the electricity burned while gaming, your system is limited by your GPU in both instances. If you have an RTX 5090 then you can throw almost 600 watts continuous at both which could use like 14kW/day, but most people will have more modest hardware, and therefore the energy usage of running a local model wouldn't be all that much either - I don't think many people limit their gaming because of how much the electricity would cost.

But when you have a datacenter with hundreds of servers capable of maxing out the residential house mains of half a dozen houses each (single NVIDIA NVL72 AI server has 72 GPUs and uses 120-150kW at full load) they can throw way, way more compute at you than you ever could manage locally.

Why is that? The electricity burned is about the same, no? Of course, the impact is more distributed.

Forget about everything related to the prompt itself being processed for the moment. What other information do you think they are grabbing and processing along with the prompt? Maybe some data about you and your interactions for resale? Account info? Piecing together details about you perhaps, for some of that highly-targeted-advertising-data companies love to sell?

Even the comment you are replying to, IMO, is pretty conservative about the actual energy savings.