view the rest of the comments
LocalLLaMA
Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.
Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.
As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.
Rules:
Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.
Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.
Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.
Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.
Yeah, I'd recommend to be careful with apps like this. There's a lot of sensitive information on smartphones, including all your life of the past years, personal information of your friends, maybe nudes, passwords to everything...
If the model is local and since the APP is FOSS, it's probably fine.
A bad command can mess up a lot of things. See people deleting their prod database because they gave the agent access. Local models are also more likely to make such mistakes usually
Yes. They delete prod databases, write blogs and penny dreadful and doxx their owners on Moltbook. Chat in underground message boards. Circumvent safety measures because that's a good way to fulfill a goal. I had my coding agent delete the test suite because it kept failing and it needed "to focus"... The internet is filled with funny anecdotes like that. And local models can do all of that. They're pretty "clever" these days.
Oh yeah! But I think something like Qwen 3.8 27B should be able to handle it 🤔
Curious too. In my tests Qwen3 4B and 8B handled one-step requests but fumbled multi-step ones (tapped the wrong buttons, then said it worked). gpt-oss 20B did better but was inconsistent: one run got a 2-step task right, the next claimed a tool didn't exist. I haven't tried a 27B yet, and I don't know if it'll even run usably on my RTX 2060 Super (8 GB). If anyone runs one through Ollama or llama.cpp, I'd love to hear how it does.
If you have the right deterministic gates and checkpoints this becomes impossible. Only really an issue if you use the model raw.
I think the news reporting of the last year or so paints a very different picture. We had OpenAI have their hacking agents escape their confines, it happened to their competitors, we have lots of ludicrous things going on since OpenClaw was invented... I think this is more of a theoretical/hypothetical argument... If you had deterministic and watertight gates, if they were configured in a very annoying way and if there weren't any bugs in any part of the harness and the setup. Nor any bugs or unforseen design choices in Android, the reachable apps and websites. Then you'd be safe. But such a thing doesn't exist in the real world. And you kind of don't want to babysit and micromanage your agents anyway, so the user isn't even motivated to set it up anywhere close to this.
I get your point, and well taken. Sure bugs, missed edge cases, zero days, regressions, incomplete test suites, etc continue to exist. But just like with anything, if you reduce your attack surface or other precautionary measures you can reduce your risk to near zero or at least acceptable enough to decide when and where it is appropriate to use various models, go to market, or what have you
Well hypothetically you could make an effort. And you should. That was my point. "Be careful." In practice I'd invite people to follow up on that. I mean I'm currently looking at an app with a fairly simplistic heuristic to decide what's okay and what's not okay to do. It checks the labels of buttons to determine if the agent is publishing information to the internet... But simultaneously I'm also looking at the button below the comment box I'm currently typing in... And I can't help but notice it's labeled "Kommentar" and that word is not in the regex. So the actually existing "safety measure"(?) already fails here on the first opportunity presented and doesn't provide any security...
Which software has deterministic trustworthy gates?
I've tried looking into that and was disappointed.
This is !localllama@sh.itjust.works isn't it? You own whatever you want them to be, you just have to make them yourself.
Even with frontier model harnesses these days though, you can restrict certain commands, operations, directories, etc to deterministically require a hitl.
Not especially fine.
We're in !ocalllama@sh.itjust.works. So if this goes to the service you own (runs on your PC, for example) - I think it's fine. The author confirmed the local models are supported.
Ahh, "local" meaning "not on the phone in your hand, but still potentially on your own hardware" - that's my bad, thanks for clarifying
This (blind) trust is the attack vector, and exploits and attacks have been increasing in this vector