Log Collection (lemmy.procrastinati.org)

submitted 3 years ago by SeeJayEmm@lemmy.procrastinati.org to c/selfhosted@lemmy.world

13 comments fedilink hide all child comments

I'm in the process of re-configuring my home lab and would like to get some help figuring out log collection. My setup was a hodgepodge of systems/OSes using rsyslog to send syslogs to a syslog listener on my qnap but that's not going to work anymore (partly because the qnap is gone).

My end-goal is going to be as homogeneous as I can manage. Mostly Debian 12 systems (phy and vm) and Docker containers. Does anyone know of a FOSS solution that can ingest journald, syslog, and if it's even possible to send docker logs to a log collector?

Thanks

you are viewing a single comment's thread
view the rest of the comments

[-] tko@tkohhh.social 5 points 3 years ago

I use a Graylog/Opensearch/Mongodb stack to log everything. I spent a good amount of time writing parsers for each source, but the benefit is that everything is normalized to make searching easier. I'm happy with it as a solution!

[-] vegetaaaaaaa@lemmy.world 2 points 3 years ago* (last edited 3 years ago)

I also use graylog to aggregate logs from various devices (mostly from rsyslog over SSL/TLS). The only downsides for me are the license (not a big problem for a personal setup), and resource usage of the general graylog/elasticsearch stack. I still think it's great.

I use this ansible role to install and manage it.

For simpler setups with resource constraints, I would simply use a rsyslog server as aggregator instead of graylog, and lnav for the analysis/filtering/parsing part

this post was submitted on 25 Aug 2023

20 points (91.7% liked)

Selfhosted

61853 readers

397 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

Be civil.
No spam.
Posts are to be related to self-hosting.
Don't duplicate the full text of your blog or readme if you're providing a link.
Submission headline should match the article title.
No trolling.
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world