My goal is once we are approved to start accepting donations that I can purchase dedicated hardware for this instance. I'd get a used server at about $2300 which would be sufficient a good amount of extra users and through it into its own dedicated shared colo at about $100/month. Factor in about $300-400 a year for drive replacements and we are left with $2300 / 12 month= 191.66 + 100/month for the shared 1u colo + a budget of $400 for drive failures throughout the year $33/month. 191.66 + 100 + 33 = $324.66/month for the first year dropping to about $133 per month after the first 12 months. It's worth noting that this method would give us double the amount of resources and quite a bit of extra storage.

Ideally we don't keep this instance on a single server forever and start to think about spreading it over multiple hosts at or after around 100K users (or less if the number of active users is high).

If someone wanted to host an instance they would not need to allocate as much resources as I have to this instance and depending on how active the instance gets could run off something a lot less powerful.

[–] ScrawnyStork@sh.itjust.works 23 points 1 year ago (1 children)

I really appreciate the information, it's very interesting to me. Given that you have a fairly specific price in mind for a server, what kind of hardware are you thinking of?

[–] TheDude@sh.itjust.works 24 points 1 year ago (2 children)

Something with Dual CPUs, at least 128GB ram, dual 750W PSUs, hardware raid (12Gbps) and 8 x 2.5" SAS/SATA slots for SSD Drives on a raid 10

[–] Shit@sh.itjust.works 7 points 1 year ago (1 children)

Ddr4 or 5? I might have 128gb of rdimm ddr4 I'm never going to use sitting on my desk.

[–] your_mind_aches@sh.itjust.works 6 points 1 year ago (1 children)

ECC memory though?

[–] Shit@sh.itjust.works 3 points 1 year ago* (last edited 1 year ago)

Yeah registered dimm 4x32g. I couldn't find a way to put it in a low powered quiet box without spending a bunch of money. My home lab stuff is ddr5 now.

[–] humancrayon@sh.itjust.works 2 points 1 year ago* (last edited 1 year ago) (1 children)

This sounds exactly like the poweredge r530 I have in my homelab. Managed to snag it on eBay with those specs, minus drives, for $350.

[–] TheDude@sh.itjust.works 4 points 1 year ago

The 530 is 2U, the 630 is 1U size factor. Looking more at the 630

[–] imaqtpie@sh.itjust.works 21 points 1 year ago (1 children)

I love the transparency. I think we can easily reach that mark. Whenever you get approved for donations we'll be ready. I've got at least tree fiddy in my account

[–] thelsim@sh.itjust.works 8 points 1 year ago* (last edited 1 year ago)

I agree. Thanks for keeping us updated and explaining what kind of money needs to be spend.
I’ll be more than happy to chip in.
edited because Thelsim doesn’t know how to proofread

[–] ComfortablyGlum@sh.itjust.works 15 points 1 year ago

Thank you for the money and time you put into making this instance work and keeping it working. I imagine the responsibility that comes from all this is both a joy and a burden.

[–] shectabeni@sh.itjust.works 13 points 1 year ago

Really nice breakdown thanks for sharing! Totally reasonable goals to reach too.

[–] SomeAmateur@sh.itjust.works 12 points 1 year ago* (last edited 1 year ago)

This is exactly what I was after thanks for the good rundown! Also thanks for all the time money and effort spent on all of this

[–] formatc@sh.itjust.works 7 points 1 year ago (1 children)

What are you doing for backups?

[–] baked_tea@sh.itjust.works 2 points 1 year ago (2 children)

That's the "drive replacements" part I believe

[–] BigDanishGuy@sh.itjust.works 8 points 1 year ago* (last edited 1 year ago)

Drive replacement != backup

Drive replacement = maintenance, a subset of physical security.

Backup = logical security.

The purpose of backup is to prevent loss of data in general, not only on account of drive failure, but also other sources such as malicious activity

[–] formatc@sh.itjust.works 1 points 1 year ago

And what if several drives fail at once or a bad actor deletes the data? :)

[–] CodeInvasion@sh.itjust.works 4 points 1 year ago (2 children)

Have you considered the implications of hardware failure on uptime? And where the cost to maintain a physical hardware will come from? What about scaling requirements?

I'm not a network engineer, but I've been involved in the corporate argument of Cloud vs On-Prem. hosting for years now. The costs always come out better for Cloud when factoring in other indirect costs like facilities and labor.

Granted it's always been on the scale of hundreds of millions to billions of dollars, and I haven't run the numbers on smaller requirements. I just wouldn't want to expose additional points of failure in return for slightly lower monthly costs.

[–] Wats0ns@sh.itjust.works 2 points 1 year ago (1 children)

I think the cost always come out better for cloud for a given reliability level. But this is a volunteer run thing, so we won't mind if there is some more important downtime than on reddit or Twitter. I really do think that if your objective is not reaching 100% uptime but cost reduction, then on prem really becomes the cheapest option

[–] CodeInvasion@sh.itjust.works 1 points 1 year ago (1 children)

A very good point! We don't need constant uptime. But I worry about the hidden costs of On-Prem, and worst case scenario where TheDude is on vacation somewhere and the instance crashes, it could be down for a while. It's also not a worry I would want to force on them either.

[–] Wats0ns@sh.itjust.works 2 points 1 year ago

Yes, when I try to explain this to people, I always explain the bus factor concept: how many people could get hit by a bus until it becomes critical to run your business ! Running in the cloud allows you to avoid this problem, there will always be an oncall tech in the DC of your cloud provider, which is very hard to organize for an on prem system !

I guess The dudes can always give remote access to someone he trusts, but at the end of the day if a disk fail somebody got to go switch it

[–] Ajen@sh.itjust.works 2 points 1 year ago

He mentioned colo, so it sounds like he's already decided against on-prem.