Différences
Ci-dessous, les différences entre deux révisions de la page.
Les deux révisions précédentes Révision précédente Prochaine révision | Révision précédente | ||
tuto:monitoring:graylog_to_crowdsec [05/03/2021 12:28] dani [Background] |
tuto:monitoring:graylog_to_crowdsec [05/03/2021 19:19] (Version actuelle) rv [Background] |
||
---|---|---|---|
Ligne 3: | Ligne 3: | ||
===== Background ===== | ===== Background ===== | ||
- | Crowdsec' | + | Crowdsec' |
* I really like the idea behind the Journal (systemd-journald), | * I really like the idea behind the Journal (systemd-journald), | ||
* Let's assume we have 40 VM on which we'd like crowdsec agent running. This means something like 40x80MB = 3.2GB of RAM, just for crowdsec | * Let's assume we have 40 VM on which we'd like crowdsec agent running. This means something like 40x80MB = 3.2GB of RAM, just for crowdsec | ||
Ligne 12: | Ligne 12: | ||
===== Send logs from Graylog to ? ===== | ===== Send logs from Graylog to ? ===== | ||
- | As I already have all my logs in Graylog, it'd be better to send this stream of logs to a single crowdsec installation. But, for now, crowdsec doesn' | + | As I already have all my logs in Graylog, it'd be better to send this stream of logs to a single crowdsec installation. But, for now, crowdsec doesn' |
+ | |||
+ | Here's the global flow | ||
+ | |||
+ | {{ : | ||
+ | ===== g2cs ===== | ||
+ | |||
+ | I wrote a small perl daemon, named g2cs (Graylog | ||
+ | |||
+ | <code bash> | ||
+ | perl g2cs.pl --port 514 --logdir / | ||
+ | </ | ||
+ | * Port is the port number g2cs will listen on | ||
+ | * logdir is where it'll write logs for crowdsec to consume. Inside this logdir, g2cs will create sub directories, | ||
+ | * syslog.log | ||
+ | * nginx | ||
+ | * access.log | ||
+ | * error.log | ||
+ | * httpd | ||
+ | * access.log | ||
+ | * error.log | ||
+ | * zimbra | ||
+ | * mailbox.log | ||
+ | * maxlines is the number | ||
+ | |||
+ | <note tip>You can choose a directory on a tmpfs filesystem to improve performance, | ||
+ | |||
+ | < | ||
+ | ===== Configure crowdsec ===== | ||
+ | Now that we have our g2cs daemon running, you can configure crowdsec acquisition to read these files. Something like | ||
+ | <code yaml> | ||
+ | --- | ||
+ | filenames: | ||
+ | - / | ||
+ | labels: | ||
+ | type: syslog | ||
+ | |||
+ | --- | ||
+ | filenames: | ||
+ | - / | ||
+ | - / | ||
+ | labels: | ||
+ | type: nginx | ||
+ | |||
+ | --- | ||
+ | filenames: | ||
+ | - / | ||
+ | - / | ||
+ | labels: | ||
+ | type: apache2 | ||
+ | |||
+ | --- | ||
+ | filenames: | ||
+ | - / | ||
+ | labels: | ||
+ | type: zimbra | ||
+ | </ | ||
+ | |||
+ | ===== Install the syslog output plugin on Graylog ===== | ||
+ | OK, now that we have crowdsec and g2cs ready, we need to send our logs from Graylog to g2cs. For this, we' | ||
+ | |||
+ | ===== Create a syslog output ===== | ||
+ | Now in Graylog, you can create a new output. Go in System -> Outputs. Select the "Syslog output" and click launch new output | ||
+ | |||
+ | {{ : | ||
+ | |||
+ | Now, configure your Syslog output like this : | ||
+ | |||
+ | {{ : | ||
+ | |||
+ | * Choose the UDP protocol | ||
+ | * Enter the DNS name or IP address of your server running g2cs | ||
+ | * Choose the port on which g2cs bind | ||
+ | * Choose the CEF message format | ||
+ | |||
+ | ===== Assign output to streams ===== | ||
+ | Now, you can assign in Graylog your new output to the streams you want. Go in the Stream menu, then, " | ||
+ | |||
+ | {{ : | ||
+ | |||
+ | And assign your Syslog output | ||
+ | |||
+ | {{ : | ||
+ | |||
+ | You should now see logs flowing from Graylog, to crowdsec. I'm using this on a small graylog setup, ingesting about 400msg/sec, out of which ~200msg/sec are parsed by my single crowdsec install. I just have to install the bouncers where I want to react to all the agressive IP collected on all my servers | ||
+ | |||
+ | ===== Run crowdsec with less privileges ===== | ||
+ | Bonus point : as crowdsec only access logs from the g2cs daemon, you can run both as a less privileged user, instead of root. First, create an unprivileged user | ||
+ | |||
+ | <code bash> | ||
+ | useradd -r -s / | ||
+ | </ | ||
+ | |||
+ | Now adapt the systemd unit for crowdsec, eg in / | ||
+ | |||
+ | <code ini> | ||
+ | Service] | ||
+ | User=g2cs | ||
+ | Group=g2cs | ||
+ | </ | ||
+ | |||
+ | And create a systemd unit for g2cs itself, / | ||
+ | |||
+ | <code ini> | ||
+ | [Unit] | ||
+ | Description=Graylog to Crowdsec syslog daemon | ||
+ | After=syslog.target | ||
+ | |||
+ | [Service] | ||
+ | Type=simple | ||
+ | ExecStart=/ | ||
+ | User=g2cs | ||
+ | Group=g2cs | ||
+ | Restart=always | ||
+ | PrivateTmp=yes | ||
+ | PrivateDevices=yes | ||
+ | ProtectSystem=full | ||
+ | ProtectHome=yes | ||
+ | NoNewPrivileges=yes | ||
+ | SyslogIdentifier=g2cs | ||
+ | |||
+ | # Allow binding on privileged ports | ||
+ | CapabilityBoundingSet=CAP_NET_BIND_SERVICE | ||
+ | AmbientCapabilities=CAP_NET_BIND_SERVICE | ||
+ | |||
+ | |||
+ | [Install] | ||
+ | WantedBy=multi-user.target | ||
+ | </ |