This post is primarily for the people here that are forced to work with AI. But it may be interesting for everyone.
We all here are aware how bad AI is for people and the environment. During some research I found out that this effect is most likely multiplied by pure greed (no even worse then you thought).
A little bit of context: I am currently responsible for a small R&D Team (me and some trainees) for a EU managed Services provider. Because of corpo pressure we need to evaluate AI usage.
I gave multiple models, nearly all run on 150W hardware a shot. And of course I was forced to test Claude also.
And what I found buffled me: the "frontier" top nodge best of the best model by Anthropic was the only one that did not even complete the task.
It constantly violated against clear rules. All of them about code simplicity and maintainability.
For example QWEN solved the task in about 30 minutes, one shot
And this would be the bill we would have paid for Claude without any usable result.
Having a deeper look at what happens if you let different models work on an existing codebase:
Claude will introduce complexity on every change. Wasting compute and energy.
As open source models clearly can do the job, why can those super huge big models not do the job?
And I think the answer is greed:
- more tokens more money
- lock the user into your model. Make sure you can not leave anthropic.
And that cost is really high. Not only those absurdly high token prices (factor of 100 higher then open models) it burns the employees out.
Is anyone here forced to work with AI and had made similar observations?
Luckily for me, I have some influence at my job and can minimize the AI impact in every sense. And maybe such experiments and research can convince other employers to not go down this road.
I have a RTX4090 (24GB VRAM) plus 128GB DDR4 system RAM on Ubuntu 24.04 and I am running these models locally
| Model | Speed | Context window | |
|
|
| | Qwen3.8-Flash-Next UD-Q4_K_XL — 125B/6B-active hybrid MoE, 104 GB | ~16.7 tok/s | 128k tokens | | GLM-5.3-Flash UD-IQ3_XXS — 320B/18B-active MoE, 120 GB | ~9 tok/s | 128k tokens | | Qwen3.8-27B UD-Q4_K_M — dense hybrid, 16 GB, fully on the 4090 | ~49 tok/s | 262k tokens |
The larger models don't fit on the GPU alone but they're mixture-of-experts and their active weights easily fit on the 24GB. The bottleneck is my system RAM as llama.cpp has to constantly move the experts between RAM and VRAM. I'm running older hardware, AM4 CPU, DDR4 RAM, etc. so 128GB is my limit on system memory and it's relatively slow compared to DDR5. With DDR5 I would expect a good bump in speed for Qwen 3.8 Flash Next and GLM 5.3 Flash.
I also have a Kimi subscription which I use as my daily agentic driver and use DeepSeek for random tasks (DeepSeek Flash 4.1 is really fast and cheap).
That’s great. I’m on a 4090, but only 64GB of system RAM (DDR5 though). I might give it a go.
Interesting. I didn’t know how that worked, but I might give it a try. I have a 3090 with a lot of RAM.