Yes, but I also ended up picking up a couple of AMD's discrete GPUs (and sticking them in my decade old desktop) after using it for a while. It turns out that while decode speed for MoE models is decent on Strix Halo, prefill is rather slow -- i.e. you end up having to wait a rather long time before the model starts producing text unless you keep the amount of information in context very small -- and Strix Halo is abysmally slow for dense models.

I used ollama when I started out, but ditched it in favor of just using llama.cpp directly once I was more familiar with LLMs. ollama's "modelfile" is quite annoying to deal with compared to writing a presets.ini file and I couldn't figure out how to get multimodal models from HuggingFace working with it even after spending a long time trying... not that llama.cpp has been all roses either; they break shit a lot -- e.g. this major bug affecting Strix Halo systems still needs a manual fix if you're building llama.cpp yourself -- and the documentation leaves a lot to be desired... but I can change settings like temperature per request and multi-modal from community models actually works and so on. 🤷️

What OS are you using? How do you have the RAM/VRAM ratio configured? What's your stack?

Fedora, 32GB regular/96GB VRAM (because I got OOMs on 70B models with the default config -- similar to what you encountered; may end up experimenting with this more as I try to get Qwen3.8-Flash-Next working though), llama-server + custom harness.

I still mainly run Qwen3.6 35B-A3B on it (usually llmfan46's heretic version, sometimes the stock weights) but have a lot of models downloaded for testing.

Edit: Disabling the memory split (i.e. switching back to "auto" in the bios) lets me load Qwen3.8-Flash-Next at lower quants. I've managed to get it to work up to Q4_K_M so far. I think it should be possible to run a higher quant with other techniques though, but I haven't managed it yet (as of 2026-08-28 10:37PM UTC).