158
Is anyone self-hosting their own AI?
(lemmy.nz)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 17 Oct 2026
158 points (75.8% liked)
Selfhosted
63211 readers
285 users here now
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil.
-
No spam.
-
Posts are to be related to self-hosting.
-
Don't duplicate the full text of your blog or readme if you're providing a link.
-
Submission headline should match the article title.
-
No trolling.
-
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
-
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
founded 3 years ago
MODERATORS
I use Qwen 3.8 27b Q4 with 100k context (llama.cpp and the Mistral AI harness) on a 24 GB AMD Radeon 7900 XTX and it's been great. It can handle pretty much anything I throw at it and I can let it work indepedently overnight / over days on long tasks.
How much electricity does it burn? And how much does it cost to let it run overnight? I feel like this is the same problem that the companies have, its way too expensive to use it on a lots of things and it is way cheaper to use it on someone else's server..
It uses 261 W, probably more expensive than getting a 2nd Mistral subscription with the cost of power where I live (Belgium), but it will be free heat once I have solar panels and a battery.
Thank you, that is not as bad as I expected. 👍
I have nearly the same setup, except OpenCode for the harness. I run 200k context for anything that doesn’t need vision, or 150k otherwise. Typically get about 30 t/s decode With MTP, starts at close to 800 t/s for prefill and drops to the 200s for a long context. I’d imagine higher decode speeds are possible with some tinkering, but I haven’t gotten around to it.
OpenCode recently got hacked and advises changing API keys fyi.
I get about 45 t/s after some tweaking, getting the thermals under control helped a lot; mainly by reducing the power consumption to 261 W (and downclocking a little bit).
What vision model do you use?