What does your infrastructure look like? (lemmy.world)

submitted 3 years ago by demosthenes@lemmy.world to c/selfhosted@lemmy.world

31 comments fedilink hide all child comments

In the What are YOU self-hosting? thread, there are a lot of people here who are self-hosting a huge number of applications, but there's not a lot of discussion of the platform these things run on.

What does your self-hosted infrastructure look like?

Here are some examples of more detailed questions, but I'm sure there are plenty more topics that would be interesting:

What hardware do you run on? Or do you use a data center/cloud?
Do you use containers or plain packages?
Orchestration tools like K8s or Docker Swarm?
How do you handle logs?
How about updates?
Do you have any monitoring tools you love?
Etc.

I'm starting to put together the beginning of my own homelab, and I'll definitely be starting small but I'm interested to hear what other people have done with their setups.

you are viewing a single comment's thread
view the rest of the comments

[-] rs5th@lemmy.scottlabs.io 5 points 3 years ago* (last edited 3 years ago)

My setup is a mix of on-prem and VPS.

On-Prem

Primary Cluster (24 cores, 192 GB RAM, 36 TB usable storage)
- Two Dell R610 (12 cores, 96 GB RAM each)
- vSphere 7.0, ESXi 6.7 (because processors are too old for 7.0)
- Kubernetes 1.24
  - Single controller VM
  - Two worker VMs
  - OS: Ubuntu 20.04
  - K8s Flavor: Kubeadm
- Use: Almost everything
- Storage:
  - Synology 1515 (11 TB usable, RAID 5) - vSphere datastore via NFS
  - Synology 1517 (25 TB usable, RAID 5) - Kubernetes mounts via NFS, media, general NAS stuff
Standalone Node (4 cores, 16GB RAM, 250 GB SSD)
- Lenovo M900 micro-PC
- OS: Ubuntu 22.04
- Kubernetes 1.24
- K8s Flavor: k3s
- Use: provide critical network services (DNS/DHCP) if any part of the complex cluster goes down, Frigate due to USB Coral TPU plugged in here
Networking / Other
- DNS:
  - Primary: AdGuard Home running on Standalone
  - Internal domain: BIND VM running in Primary Cluster
- Firewall: Juniper SRX 220H
- Switch: Juniper EX2200-48
- WiFi: 3x Unifi In-Wall APs
- Power:
  - UPS backing compute and storage (10-15 min runtime)
  - UPS backing networking gear (15-20 minute runtime)

VPS

Single Linode (2 cores, 4 GB RAM, 80 GB storage)
- OS: Ubuntu 22.04
- Kubernetes 1.24
- K8s Flavor: k3s
- Use: UptimeKuma to monitor on-prem infrastructure, services that can't go down due to home ISP or power issues (like family RocketChat).

Every service (except Plex) is containerized and running in Kubernetes. Plex will be migrated soon(TM). Everything in Kubernetes is handled via Infrastructure as Code using FluxCD and GitOps principles. Secrets are stored in git using Mozilla SOPS for encrypt/decrypt. Git repos are currently hosted in GitHub, but I'm considering Gitea, though that might present a bit of a bootstrapping problem if all the infrastructure that hosts Gitea is declared inside Gitea...

[-] sneakyninjapants@sh.itjust.works 1 points 3 years ago

Git repos are currently hosted in GitHub, but I’m considering Gitea ...

I have a similar-ish setup to this and landed on having Gitea hosted outside the cluster using plain docker-compose along with renovate for dependency updates within the cluster. These two containers configuration files live on Github and have Github's renovate-bot keep them up to date. Changes get picked up through polling the git repo and applied to the host through Portainer, though I'm planning to swap that out with a slightly more complex solution relying solely on Gitea's new CI/CD tooling.

this post was submitted on 14 Jun 2023

19 points (100.0% liked)

Selfhosted

62819 readers

533 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

Be civil.
No spam.
Posts are to be related to self-hosting.
Don't duplicate the full text of your blog or readme if you're providing a link.
Submission headline should match the article title.
No trolling.
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world