Okay, so that VRAM Vs RAM recommendation is a bit of BS. You absolutely do not need more than the base 512MB assigned to the GPU, as thanks to UMA and GTT, the GPU can actually access and utilise system RAM.

I personally didn't splurge for 395 with 128GB RAM, instead have two nodes:

• a Minisforum UM890 Pro with Ryzen 8945HS and 64GB DDR5. The main downside of this APU is the relatively low power NPU - although thanks to open-xdna, it is becoming more and more usable, especially if you want full TTS/STT pipelines (those can run on the NPU, leaving the CPU and GPU usable for LLMs).
• a Minisforum N5 Pro for NAS. Ryzen AI 370 + 64GB DDR5 as well. Thanks to the NPU, I can pin two smaller models (12-20b at Q4_M_K) and run them parallel without impacting other services, or one large model

Lemonade runs quite well in Docker, allowing for much cleaner updates - and the ability to use a barebones OS if need be - and as the Lemonade SDK guys went out of their way to provide API compatibility with llamacpp, OpenAI, Ollama and even Anthropic, plus an MCP server, it is now incredibly easy to hook it up to any downstream wrapper - say, OmniRoute, or Home Assistant, or direct into VSCode.

Lemonade being better than Ollama

Practically any modern runtime is better than Ollama at this point. Ollama is an expensive wrapper around llamacpp. Lemonade uses a bunch of runtimes - llamacpp, vLLM, FastFlowLM, plus ROCm environments for audiovisual genAI (TTS, STT, image, video and sound generation) - to provide you an all in one solution regardless what model you want to run.

One drawback is that dependency updates can be slow at times - meaning if a new model drops that requires a super recent runtime, you’re SOL.

But then again, it contains everything needed to start deploying LLMs - all you need is a base OS with a relatively recent kernel, AMD drivers and Linux-firmware packages, plus of course Docker. I’ve actually been utilising self hosted models to keep my Docker compose setup up to date - to research changes between versions, update the config accordingly, and verify any further one-shot upgrade needs.

The NPU doesn’t really need any extra setup if you run Lemonade in Docker. The Linux kernel ships with up to date amdxdna base drivers since 6.19, and the rest is in the firmware and ROCm runtime, which can be updated separately. An up to date Ubuntu or Debian server, and the latest Lemonade docker image should have it working - all you need to do is pass through the right device /dev/accel. Then download the NPU runtimes in the Lemonade web interface, download FLM models (regular GGUF doesn’t work, so you’re limited to models converted to what FastFlowLM expects), and you’re golden.

FLM models in my experience are a bit less performant (about 20% slower tpks, haven’t measured other metrics like TTFT etc), but at the time use a fraction of power compared to the GPU or CPU - 1-2W compared to the up to ~60W max TDP of the latter two combined… and it might be placebo but I’ve also noticed that the same model FLM vs GGUF, FLM tends to have slightly higher quality responses.