20
Log Collection (lemmy.procrastinati.org)
submitted 3 years ago by SeeJayEmm@lemmy.procrastinati.org to c/selfhosted@lemmy.world
13 comments fedilink hide all child comments

I'm in the process of re-configuring my home lab and would like to get some help figuring out log collection. My setup was a hodgepodge of systems/OSes using rsyslog to send syslogs to a syslog listener on my qnap but that's not going to work anymore (partly because the qnap is gone).

My end-goal is going to be as homogeneous as I can manage. Mostly Debian 12 systems (phy and vm) and Docker containers. Does anyone know of a FOSS solution that can ingest journald, syslog, and if it's even possible to send docker logs to a log collector?

Thanks

you are viewing a single comment's thread
view the rest of the comments
[-] Dogeek@sh.itjust.works 7 points 3 years ago

You could use grafana loki to handle logs, it's similar to Prometheus so if you're already using that and/or grafana it's an easy setup and the API is really simple too.

[-] farcaller@fstab.sh 3 points 3 years ago

I second this. Loki for logs, VictoriaMetrics for metrics―it's significantly more lightweight than ELK logging (and any lags are irrelevant for a homelab), and VM is similarly much more careful with RAM than Prometheus.

[-] SeeJayEmm@lemmy.procrastinati.org 2 points 3 years ago

I did stumble on Grafana Loki in my search. Was trying to figure out if it was overkill. Is it fairly lightweight?

[-] keyez@lemmy.world 2 points 3 years ago* (last edited 3 years ago)

Much less intensive than an elasticsearch anything. I have Loki, grafana and 3 promtail clients running for my env (switched from Graylog/elasticsearch) and over the last few days Loki is sitting at 3GB memory and 8% CPU processing logs for about 6 devices.

[-] SeeJayEmm@lemmy.procrastinati.org 1 points 3 years ago

Ok thanks. Looks like I can give it a try in docker.

this post was submitted on 25 Aug 2023
20 points (91.7% liked)

Selfhosted

61853 readers
403 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

  1. Be civil.

  2. No spam.

  3. Posts are to be related to self-hosting.

  4. Don't duplicate the full text of your blog or readme if you're providing a link.

  5. Submission headline should match the article title.

  6. No trolling.

  7. Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.

  8. AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
ayyy@sh.itjust.works
curbstickle@anarchist.nexus
curbstickle_lw@lemmy.world