23
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 01 Nov 2026
23 points (92.6% liked)
LocalLLaMA
5579 readers
123 users here now
Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.
Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.
As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.
Rules:
Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.
Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.
Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.
Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.
founded 3 years ago
MODERATORS
Thanks for the very rich reply. I'm a bit reassured that I can do some basic stuff, I'm not a poweruser anyway, if this word is still used. The aim is to not rely on external tools and actors as much as possible for private stuff. My work does give me access to a commercial gpt for work, but I don't want to use it for my private life matters. The learning curve seems a bit higher that I thought, but I saw some tools that could maybe help me there, like ollama or localai.io, any thoughts on these projects?
Friends don't let friends use Ollama
If you want a gentle start, may I suggest Jan.ai? It's a ride on lawn mower compared to the sports car I suggested...but you need to dip your toe into the water before you go ludicrous speed.
Once you get a feel for that, I'm going to suggest some MCP tools to you as well. I don't know if Jan finally added MCP servers to their stack (they should have by now!).
The following are ... ok. Not the best and not what I run with but again, lawn mower before sports car
https://github.com/ofershap/mcp-server-markdown
https://github.com/jeffreyrampineda/kiwix-wiki-mcp-server
https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem
https://github.com/exa-labs/exa-mcp-server
https://exa.ai/
https://www.home-assistant.io/integrations/mcp_server/
Sorry - again: too much info. Start with Jan. You'll be amazed...and then perplexed...and then frustrated. That's the time to migrate thru to llama.cpp or PolyStrata and start adding those MCP tools I mentioned above.
https://github.com/VecSzn/PolyStrata
https://github.com/srware-net/LocalLightChat/
https://github.com/open-webui/open-webui
Then we can get freaky :) Oh...we have such sights to show you lol
Many thanks! I'll pin your reply for when I'm down the rabbit hole! Jan looks nice
Enjoy and good luck!