79
submitted 1 day ago* (last edited 1 day ago) by ImgurRefugee114@reddthat.com to c/localllama@sh.itjust.works

Local models are meaningfully useful* for agentic work now. This is a realization I came to over the last day or so.

I wrote a lot of custom mods for a game that recently switched engines. The migration is going rather smooth for most people because of wise architectural choices from the dev early on, and because they prioritized backwards compatibility in the update. But there are still some rough edges, especially if you had your hooks deep into the engine.

I recently vibeforked Unbiased Bob's fork of Strata to make it compatible, easy to use and optimized for my OS and hardware (NixOS 5800x3d 64gb, 7900xtx 24gb). Sol 6.1 got it working with Qwen 3.8 Flash Next IQ3 XSS (~75GB) at roughly 46tps 164k ctx with enough headroom to launch a heavy browser along with a triple-A game set to low in 720p for testing when I need to (plus space for zram swap)

I set it up in deepseek harness, gave it a 14MB error log in the game's working directory, then went to sleep. I woke up and it had run for 3 hours. The game launched with only two errors, which it quickly fixed from the new, vastly smaller logfile.

It hammered down every issue, making changes across 30 separate mod repos. Practically a one-shot, and all the features worked as originally intended.

I left a local coding agent running completely on its own for three hours across multiple repos across a dozen auto-compactions with a log file and no ability for it to verify its work besides a compiler. And it produced meaningful results. I can't speak to the code quality, but it worked. This was completely unthinkable even just two years ago. Even one year ago, the cloud frontier wouldn't be able to do this on its own, and now a local model can, running on my consumer gaming hardware from one or two gens back.

“Huh... So... We're here now..." I said to myself, with a bit of stunned disbelief, wondering what the future one or two years from now might hold.

top 50 comments
sorted by: hot top new old
[-] Tundra@sh.itjust.works 5 points 7 hours ago

Its nice to read something positive about AI, I used a model to get Stalker 2 running smoothly (using optiscaler)

I recently got a laptop GPU with 16gbs Vram - what models do you guys recommend?

Ive been testing this: https://ollama.com/edtorre/qwen3.5-hermes

[-] pech@lemmy.world 2 points 2 hours ago

This person on huggingface makes excellent space saving models using APEX quantizations. They produced this one to fit on gpus with 16gb of vram or less.

https://huggingface.co/mudler/Qwen3.8-27B-APEX-GGUF

[-] ImgurRefugee114@reddthat.com 2 points 4 hours ago* (last edited 4 hours ago)

Really depends on how much headroom you need and your system ram and the PCIe / vram speed because I got a friend with a pcie5 16gb gpu and 64gb of system ram that's getting slightly better performance than me using the same setup, despite half the vram.

If you want speed and not a ton of intelligence, the 3.6 30B A3B runs fast. If you need smarts, then either a qwen 3.8 27b (probably a thinking cap finetune) or try flash next (swift finetune) in unbiased strata if you don't mind taking a risk and burning some bandwidth on a 75-85GB model that might not run great. In either case, be sure to use MTP and the model card inference params

[-] pech@lemmy.world 3 points 15 hours ago

How did you configure DeepSeek harness? I'm guessing a flake would be useful.

I read your post and ended up finding this repo: https://github.com/numtide/llm-agents.nix

I'm considering deploying it for testing in a nix container. 🤔

[-] ImgurRefugee114@reddthat.com 4 points 15 hours ago* (last edited 15 hours ago)

I built my custom solution with a shared systemd qemu VM service to run harnesses isolated from my host that mount specific directories and run via wrappers that start and automatically SSH into them, as well as native variants that run on the host machine. (Plus args for quick selection and feature toggling e.g. "cx sol high memory" as well as per-harness hooks for my agnostic skill-like text injection system, memory tools, optimized system prompts, etc)

All of this is done in my nix flake (but can't share here)

For Deepseek itself, I'm just using Moraxyc/deepseek-harness.nix for the package, and using its webui, with wrappers to integrate it with my vm/native dichotomy system.

I haven't done any deep tooling integration or prompt rewriting with dsh like I have with codex and claude code. It's still pretty early days for me using it and I prefer terminal apps in a multiplexer like herdr. But it doesn't seem to need it so much either: sonnet eating 35k tokens for a "hello" out of the box was egregious to me, but dsh seems to be more light so I haven't felt the need, and I don't do as much with it and haven't used it for very long yet so I haven't felt the need to customize it as much as I have the other harnesses.... Which is a bit ironic given that its customization and flexibility are pretty much its main selling points.

[-] pech@lemmy.world 2 points 1 hour ago

Oh that sounds rad lol. I was considering the possibility of building a setup within nixos using vfio to declare and configure VMs. I like your integration with systemd, that's clever

[-] ImgurRefugee114@reddthat.com 1 points 57 minutes ago

Thanks. And yeah, the wrappers start the service if it isn't already up, then it either times out after some period when the VM isn't in use and shuts down, or I close it manually if I want to free up the resources. But it's pretty lightweight.

[-] qaz@lemmy.world 10 points 23 hours ago* (last edited 20 hours ago)

I've been following r/decomps recently. Several times a day someone posts a new AI-generated decompile, seemingly by AI agents sent off on a mission without any guidance, effort, or proper setup. I expected the tools / harnesses to develop first before seeing results like this. It's insane that models have these abilities while still being based on what is fundamentally just text generation.

[-] Wirlocke@lemmy.blahaj.zone 11 points 18 hours ago

Reminder that the transformer architecture, the thing all LLMs rely on, was originally built to translate between languages for the purpose of Google Translate.

And compilers are just translators, translating from a programming language (and/or an intermediary) into machine code. So the task of translating it back is very close to what LLMs specialize in.

So I always had it in the back of my mind that LLMs would be incredibly powerful for decompilation, I'm just surprised it took this long to be good enough to be a problem.

[-] Doomsider@lemmy.world 5 points 9 hours ago* (last edited 9 hours ago)

Thanks for that. This makes me realize in the future the only way to protect your code is to prevent it from being stored or executed locally. Software has finally been set free and corporations will have to control computing to continue their imaginary property regime.

It is going to be a wild ride for sure. I think others have already predicted a future where your access to all computing becomes a thin lan client.

[-] dermanus@lemmy.ca 13 points 1 day ago

I had a similar experience this week. I tried ollama like last year and thought it was kinda cool but ultimately not that helpful.

Over the weekend I got Qwen3.6 35b running Hermes on my gaming PC and it made me a whole back testing application for trading. Everything from planning out the work, breaking it into tasks, spinning up agents for blocks of work, writing tests, the whole shebang.

It took like two work days of me prodding it once in awhile but it's still impressive

load more comments
view more: next ›
this post was submitted on 27 Oct 2026
79 points (89.1% liked)

LocalLLaMA

5451 readers
209 users here now

Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.

Rules:

Rule 1 - No harassment or personal character attacks of community members. I.E no namecalling, no generalizing entire groups of people that make up our community, no baseless personal insults.

Rule 2 - No comparing artificial intelligence/machine learning models to cryptocurrency. I.E no comparing the usefulness of models to that of NFTs, no comparing the resource usage required to train a model is anything close to maintaining a blockchain/ mining for crypto, no implying its just a fad/bubble that will leave people with nothing of value when it burst.

Rule 3 - No comparing artificial intelligence/machine learning to simple text prediction algorithms. I.E statements such as "llms are basically just simple text predictions like what your phone keyboard autocorrect uses, and they're still using the same algorithms since <over 10 years ago>.

Rule 4 - No implying that models are devoid of purpose or potential for enriching peoples lives.

founded 3 years ago
MODERATORS