tuto:monitoring:graylog_to_crowdsec

Ceci est une ancienne révision du document !


Bridge Graylog and Crodwsec

Crowdsec's architecture allows running several agents, each parsing the local logs on the server it's running, and sending events to a local API. While this approach works and is flexible, it might not be the most efficient. In my case, all my server are already sending their logs to a Graylog instance. Running one crowdsec agent on all of those VM would be a waist :

  • I really like the idea behind the Journal (systemd-journald), it's very conveniant. But it has a major drawback : it's slow has hell ! Better have everything on SSD, or reading the journal will slows everything down. As I already have journalbeat collecting logs from the Journal, I prefer not adding another Journal reader, which will slows things down even further
  • Let's assume we have 40 VM on which we'd like crowdsec agent running. This means something like 40×80 = 3.2GB of RAM, just for crowdsec
  • I have to manage crowdsec conf on those 40 VM. Of course, ansible to the rescue, it wouldn't require manual config everywhere, but I'd still have to setup which logs to parse on which VM, which scenario to apply etc.

So, I looked for an alternative setup, and here's what I came with :-)

As I already have all my logs in Graylog, it'd be better to send this stream of logs to a single crowdsec installation. But, for now, crowdsec doesn't have network logs input, it can only reads files and the Journal (I've opened a ticket for this). So, the idea is to somehow forward the logs I want fro Graylog to a small daemon, which would write logs for crowdsec to consume. Graylog supports forwarding logs using GELF out of the box. But I decided to use the syslog output plugin instead. You just have to put the jar file in your Graylog plugins dir, and restart Graylog. Then, you can create a new Syslog output

  • tuto/monitoring/graylog_to_crowdsec.1614943673.txt.gz
  • Dernière modification: 05/03/2021 12:27
  • de dani