84
submitted 1 day ago by CAVOK@lemmy.world to c/europe@feddit.org
you are viewing a single comment's thread
view the rest of the comments
[-] Sibbo@sopuli.xyz 10 points 1 day ago

Cool! How much does a home server cost that runs it at a reasonable tokens per second?

[-] ramble81@lemmy.zip 14 points 1 day ago

If you want a more reasonable European developed open-weight model, consider looking at Ministral. Developed by the French, and it’s a slightly smaller version of their Mistral full. Ministral 3 - 27B runs just fine on a 12GB card.

[-] gnutrino@programming.dev 18 points 1 day ago

Running EUROPA on in-house hardware would take a lot of memory, even with the usual compression to 4 bits per parameter. At that level, its weights alone would take up at least 476 GB by our math, plus working memory. That puts it in the same range as Muse Spark Open, Meta’s open model. That model has 816 billion parameters and runs on two Apple Mac Studio desktops with 512 GB of memory each.

Although it's worth noting they haven't actually trained the thing yet so we'd have to wait to get the actual weights to know more.

[-] OhmeHose@feddit.org 5 points 1 day ago

Judging from the size of other frontier models on hf, you won't host it.

The best/biggest you can squeeze on 64gb ram rn is ~70-80B in a Q4 Quant. Going lower with quantization will usually leave you with notable degraded performance.

[-] massive_bereavement@fedia.io 3 points 1 day ago

Is that 64gb vram or ram, cuz if it's the earlier I guess that's priced like a cheap car.

[-] OhmeHose@feddit.org 3 points 1 day ago

Depends on your hardware. Apples unified memory system can give near complete access to the ram for their GPU/npu.

Apart from that you don't need a single bit of vram/gpu to host a model. Then can be run completely on CPU+ram. But that comes with a significant processing penalty.

On top of that there's new "engines" which can run a LLM from disk storage +system ram/vram, yet again with lots of speed penalities, since you chunk the model into small pieces and load them into ram accordingly. This will also cripple the lifetime of ssds with the sheer amount of read/write cycles.

Then there's also dedicated systems or extension cards which can be used as accelerator.

None the less we speak about a minimum of 5000€'s for a system which can run llms and (lightweight) games simultaneously.

[-] B0rax@feddit.org 1 points 1 day ago

VRAM of course

[-] Sibbo@sopuli.xyz 1 points 1 day ago

Is 64gb the limit of consumer GPUs at the moment?

[-] B0rax@feddit.org 6 points 1 day ago

You can also get systems with unified memory (from amd for example, or from Apple), which will be available* up to 512gb of ram.

*that means, if you pay the equivalent of a middle sized new car.

[-] OhmeHose@feddit.org 3 points 1 day ago

Best bang for the buck is a Mac mini with 64gbs memory. But that's still ~3k€

[-] Sibbo@sopuli.xyz 1 points 1 day ago
[-] B0rax@feddit.org 2 points 1 day ago

Yes. A 512gb ram Mac Studio costs 25k€

[-] fairkom@fairmove.net 2 points 1 day ago

@Sibbo @CAVOK Our colleague @rasos compiled a PC for €2000.- with second hand parts including a RTX 3090 with 24GB VRAM. Qwen3.7 is dojng an amazing job with 40-50 t/s see his blog https://roland.alton.at/roland/rolog/ki-selbst-betreiben

https://roland.alton.at/roland/rolog/ki-selbst-betreiben

[-] Skipcast@lemmy.world 3 points 1 day ago

Domyn has not yet released EUROPA’s weights, the files that hold its parameters and let anyone run it on their own hardware. It plans to publish them with details of its training methods when EUROPA comes out. The post gives no date. In June, the outlet Declic Media reported that Domyn was aiming for a release “within the year.”

Guess we'll have to wait and see

this post was submitted on 15 Nov 2026
84 points (95.7% liked)

Europe

12486 readers
369 users here now

News and information from Europe 🇪🇺

(Current banner: La Mancha, Spain. Feel free to post submissions for banner images.)

Rules

  1. This is an English-language community. Comments should be in English. Posts can link to non-English news sources when providing a full-text translation in the post description. Automated translations are fine, as long as they don't overly distort the content.
  2. No links to misinformation or commercial advertising. When you post outdated/historic articles, add the year of publication to the post title. Infographics must include a source and a year of creation; if possible, also provide a link to the source.
  3. Be kind to each other, and argue in good faith. Don't post direct insults nor disrespectful and condescending comments. Don't troll nor incite hatred. Don't look for novel argumentation strategies at Wikipedia's List of fallacies.
  4. No bigotry, sexism, racism, antisemitism, islamophobia, dehumanization of minorities, or glorification of National Socialism. We follow German law; don't question the statehood of Israel.
  5. Be the signal, not the noise: Strive to post insightful comments. Add "/s" when you're being sarcastic (and don't use it to break rule no. 3).
  6. If you link to paywalled information, please provide also a link to a freely available archived version. Alternatively, try to find a different source.
  7. Light-hearted content, memes, and posts about your European everyday belong in other communities.
  8. Don't evade bans. If we notice ban evasion, that will result in a permanent ban for all the accounts we can associate with you.
  9. No posts linking to speculative reporting about ongoing events with unclear backgrounds. Please wait at least 12 hours. (E.g., do not post breathless reporting on an ongoing terror attack.)
  10. Always provide context with posts: Don't post uncontextualized images or videos, and don't start discussions without giving some context first.

(This list may get expanded as necessary.)

Posts that link to the following sources will be removed

Unless they're the only sources, please also avoid The Sun, Daily Mail, any "thinktank" type organization, and non-Lemmy social media (incl. Substack). Don't link to Twitter directly, instead use xcancel.com. For Reddit, use old:reddit:com

(Lists may get expanded as necessary.)

Ban lengths, etc.

We will use some leeway to decide whether to remove a comment.

If need be, there are also bans: 3 days for lighter offenses, 7 or 14 days for bigger offenses, and permanent bans for people who don't show any willingness to participate productively. If we think the ban reason is obvious, we may not specifically write to you.

If you want to protest a removal or ban, feel free to write privately to the admin that applied the rule (check modlog first to find who was it.)

founded 2 years ago
MODERATORS