87
Even easier than using LMStudio? I wonder how they want to manage that.
I keep looking at mac mini's and wondering if I should run models locally. But they're always so expensive I don't really need to run AI models at all it's just an idle curiosity.
I think I'm going to ignore AI until it either gets cheaper or builds us a warp drive.
Them being a great deal is gone
Only 3 years behind. Getting better, Apple
Thyey aren't 3 years behind on custom hardware to run local AI models.
E.g nvidia's DGX Spark came out in late 2025.
Their MacBook pros have been running models for years just fine, but this is more purpose built.
Edit: Also the Studio Ultra's have been running AI models for awhile as well with their higher memory bandwidth and total memory allowed, but they're very expensive.
I mean it's apple and their AI is Siri. They're lucky if they can even break even.
Harbor is currently in a private preview under Apple’s Claris brand...
Now there’s a name I haven’t heard in a long time.
FileMaker is still going strong
I give it a week before someone reverse engineers it and uploads a foss option.
Idk if I hate this kind of corpo project if they’re paying people for their work over more FOSS-slop
There are already a lot of FOSS options that run on Apple Silicon.
Oh sure I'd imagine so. What I meant was whatever magic sauce that makes this better will be assimilated.
Sooooo it’s Ollama but not FOSS?
https://sleepingrobots.com/dreams/stop-using-ollama/
Turns out that ollama is kinda garbage, and their desktop app wasn't FOSS for a while too.
Also, modelfiles suck
Ooh I didn’t know that! Thank you! I like FOSS, so I’ll try out the stuff referenced in your article!
Call me when something lets me train and run my own 1.4T model locally, in about 120 years.
Those 120 years went by in a flash.
He is the world's dumbest man afterall
I feel there is a catch here. I forgot mixture of experts was a thing -_-. They only activate like 32B at a time.
But this can be incredibly good, lets see!
Btw "and train" people missed the very very key word.
EDIT: @_@ training qwen 3.8 on under 6 GB. Yes, this seems VERY acceptable! I can see the endless potential here.
Nah, it seems it can finetune a 128 GB model with 6 GB RAM, I don't see evidence that it can finetune something of 1.4T or more. I was excited for nothing.
Maybe just another two years, but the trajectory is promising and hardware will improve, as well. Strata is good for inference, at least.
sus
it is called install asahi or buy a proper computer
Dynamic pooling could make this more accessible to lower power macs.
Are there other handlers that can bundle computers into a local Ai instance in a similar manner?
I guess vLLM supports clusters so you could just use that? The setup is more difficult than just plugging in a cable though.
Interesting. I kind of want to buy another Mac to dee if it works either way my M4 Max MBP.
this post was submitted on 11 Nov 2026
87 points (82.7% liked)
Technology
89787 readers
6114 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS