Totally new to running local LLM. How are you connecting to your model via mobile? (lemmy.world)

submitted 4 months ago by venusaur@lemmy.world to c/fosai@lemmy.world

18 comments fedilink hide all child comments

I’m connecting to llama.cpp on my laptop through my phone via Tailscale but when my laptop sleeps I can’t access it anymore on my phone.

What are yall using for this? Thanks!

you are viewing a single comment's thread
view the rest of the comments

[-] tal@lemmy.today 7 points 4 months ago* (last edited 4 months ago)

When your laptop is sleeping, it can't be doing computation. I mean, that's not specific to LLMs. You're going to have to set it in your OS to not sleep if you want to remotely access it, or to sleep under conditions that don't arise for you (e.g. "don't sleep if the lid is closed while the laptop is plugged in" or whatever).

The only exception I can think of would be if you (a) have it connected to wired Ethernet and (b) the laptop can do Wake-on-LAN; in that scenario, you could rig something up where it wakes up when something tries to contact it. In practice, you probably just don't want to have it sleeping.

[-] venusaur@lemmy.world 2 points 4 months ago

Thanks! Have you ever set up wake on LAN? Any recommendations?

[-] tal@lemmy.today 1 points 4 months ago

Have you ever set up wake on LAN?

Nope, sorry. Also, based on the WP article, apparently there's now a wireless variant of Wake-on-LAN that I'd never heard of until now (again, assuming that the wireless chipset and the system supports it).

[-] venusaur@lemmy.world 1 points 4 months ago

Thanks! I’ll do some reading.

[-] khannie@lemmy.world 1 points 4 months ago

I did, approximately forever ago. I recall having to change something in the bios and some minor issues generating the "magic packet" but overall it was less hassle than I expected. This was for a desktop.