OpenAI buying tens of thousands of Macs for AI is the funniest possible plot twist: spend years convincing everyone the cloud is the future, then discover the hottest AI infrastructure is basically a warehouse full of Mac minis. Somewhere, an IT admin is whispering: »I told you so.«
I’d be interested in buying one as a headless server, but a lot of the FLOSS software I’d like to run seem to prefer x86 processors.
Truly? Even with their translation layer? I forgot what's it called.
The shared memory for cpu and gpu was a cost saving thing, now it's a massive help as nvidia limit consumer gpus to 24gb
The translation later is called Rosetta 2 (Rosetta 1 having been for the PowerPC to Intel transition), and it looks like it's mostly going away next year, so buying a Mac with the intention of using it would probably be a bad idea at this point.
Which really really really sucks, because Rosetta is amazing. it’s extremely performant at running x64 code on Apple Silicon, and I’ll be sad to see it go.
To Apple’s credit, though, they did extend its end of life a full year. Rosetta 2 Support was supposed to end this year with the release of macOS 27 Golden Gate, but due to enough backlash, they extended it to next year. (which they also did with Rosetta 1, back in the day.)
The real thing I’m pissed about is they won’t open-source it, which Apple sometimes does (or, used to do) with its tools.
Fwiw they claim they will support it as a framework into the future, so things Linux VMs can still take advantage of it. The only thing you're losing is desktop Mac apps which have largely all converted to arm64 at this point, except games which you probably aren't going to play natively anyway
Well— Apple wants devs to support Apple silicon. Cyberpunk 2077, for example, did very well as a native release. But a lot of that was tied to Steam’s (and GOG’s) “works anywhere“ purchase model.
And as far as “framework for the future“ are concerned, yeah, we’ll get there independently eventually, but nothing compares to Rosetta. It’s just fucking amazing, and it works now.
What exactly do you think a cloud is? It's built exactly like that, computers connected and orchestrated to run a client's arbitrary services.
Now the interesting thing here is to choose many different types of smallscale hardware. Must be some fun to administrate.
Exactly. It’s super uncommon to use standard consumer hardware to build a data center.
At this point it's pretty blatant that OpenAI are engaging in price fixing more than they are actually delivering AI datacentres.
Smells like 'Open’AI is digging another moat where there is none, like it did with the memory wafer deal that started RAMageddon. Heaven forbid people actually being able to run local models, can’t run local models if 'Open’AI has bought all the capable smaller machines, taps head. Fuck SamA and his market cornering with a rusty chainsaw. I guess they think all they have to do is keep that moat somehow until IPO and then it’s somebody else’s problem, they and their investors cash out. Dear Goddess I hope it blows up in their faces.
Ah, I see you’ve stumbled across the real reason for the “AI boom”.
Hint: it’s not because the models are SO incredible. It’s the fascists taking computers away from us so we can’t stifle them. So we can’t talk shit on billionaires anymore. So we can’t fundraise for a better world.
Just like our cars and our traffic lights watch our every move. Just like our phones have 100 sensors that profile us from the moment we wake up.
Demons took the wheel a long time ago.
Yeah, I've had the thought. 'What are we going to do with all that compute now that AI as worker replacement is a bust? Why not run panopticon and get rid of that pesky general compute in the hands of the masses?'
Centralizing compute has been tried again and again, initially mainframes, then the home PC, thin client / server, cloud compute, arguably phones (as apps processing in the cloud), Geforce Now, AI. So far it’s always gone back to the edge for most of the population, except for niche cases (or not in the case of phones, but they turned into general, if locked down, compute), good chance it won't stick this time either.
My new gaming PC that I bought about 18 months ago went up in value by 75% based on its RAM and GPU price and while I'm relieved I got it when I did, that's pure insanity that a PC would appreciate in value
I overbuilt built my NAS/container server for $1262 excluding the price of the rack chassis, PSU, cooler and two 1TB NVMe SSDs I cannot recall the price I paid for. B550 board, Ryzen 5, four 10TB Red Plus HDDs, Intel Arc A380, 64GB of DDR 5 ECC. Those listed components are now $3381. When or if you can even find them. The Arc is unobtanium as far as I can see. Of course the RAM and HDDs went up the most in cost, but it's still like $65 and $75 more for the mobo and chip.
Whoop, decided to check and I found the SSDs: I paid $60 a pop for Samsung 980 Pros and they're $240 now. So $1382 then vs $3861 now. Almost triple. The bubble can't pop soon enough.
I'm convinced that OpenAI buisness model is to create AI bubble, to sell its "reserved memory wafers" and now Macs, at the top of ram extortion pricing moment.
That same thought just occurred to me: hype a McGuffin and get investors to lend you insane amounts of money which you then use to buy up a marker, causing artificial scarcity, then sell at massive profits.
Local 27b models are good enough for most tasks.
Can’t wait to buy one of these from Ebay for 10% of the price next year.
And that's really why they're hoarding them.
No, they’re hoarding them so you have to pay for cloud services they control from now on. With your little Fire tablet. No more pirating movies or political organizing for you, piggy
Yeah no way they will allow any of this hardware to go back onto the market. Anything they dont use anymore will be destroyed.
Buy it, destroy it. Just like buying bunch of old books, train their LLM's and burn it. Humanity has gone a long way to be that stupid.
https://github.com/AtomicBot-ai/atomic-llama-cpp-turboquant
I'm running gwen 3.6 with 131k context window on a 3090, it's fast enough and about as good as pay to play Claude at work.
Upgrade that to 3.8 as soon as your hardware allows (and your use case makes sense). 3.8 is quite a bit more rational.
Any 27b Model you can currently recommend for a 16gb AMD ? Mostly coding tasks but not exclusively.
There is a way. There was a post yesterday on exactly this, let me find it... https://lemmy.world/post/51283416
9060xt 16gb is the most cost effective new GPU, but if you're going used look for a V620 on eBay. It's a 6800xt chip but in server form factor GPU with 32GB vram. Can be a bit of a pain to set up but by far the most cost effective option IMO
V620s were a good deal when you could get them for $350, now they’re $700+ and no longer a good deal.
There just aren't any good deals any more. Prices for everything have gone crazy in the last few months. For coding LLMs the cloud services may now be the least worst value, by design, until they hike the prices.
That said, I still just paid way too much for a used graphics card so I could do many things locally, because I just don't want to give the likes of Sam Altman a single penny.
Oh wow you aren't kidding. The dude I bought from on eBay @ $350 in February is sold out now. Guess I retract my statement.
This AI pricing is wrecking every deal on the market lol
cant wait for their debt calls and they have to sell their inventory to make payment.
Madoff Altman is at it again.
Intel must be kicking themselves for not using their own fabs for Arc, AND divesting from Micron, Optane, and HMC.
Apple and others must be too, for snubbing Intel and Samsung.
They would be making money hand over fist, and relieving the market at the same time.
Seems like Apple‘s little gambit to be immune to AI price fixing savagery didn’t work as well as they thought it would…
I guess Rosetta 2 was just too good…
Rosetta 2 is irrelevant here because they'd just compile their shit for ARM for better performance.
At least unlike B300s, these can one day be used if OpenAI dumps them on the used market. IF they do that instead of trashing them to make sure everything stays expensive.
Apple‘s little gambit to be immune to AI price fixing savagery
I don't think it was ever a strategy. If it was they would've stopped mac's from ever reaching OpenAI's datacenters. Or atleast they'd have prevented them from hoarding so much of the M5 production.
Has the computers for local AI become that good?
Macs, in particular, are likely to be the best option at the moment, unless you absolutely need the largest Frontier models.
Just wait for Mac m5 ultra
The Mac Studio already has it. And from October, it’ll also be available with 512GB. It does cost around 18K, though.
M5 ultra as indeed up to 512gb of unified memory. So you can run very large models. At the same time it's much much faster memory speed and bandwidth, almost simular to the nvidia rtx pro 6000.
Sure the price is also high. But you get much better value for this setup compared to what nvidia is delivering you today. I still want to see even more competition.. Not just from Apple.
sigh I just wanna upgrade my video card, man
Well, this may explain why my work computer has been on back order for the 4 months that I have had my job. Still have no idea when I’m actually going to get it.
They're the worst kind of scalpers
y'all realize this is what a hostile takeover looks like, right?
Are you talking about a stocks takeover? Because I fail to see how buying up hardware has anything to do with it.
This is a most excellent place for technology news and articles.
