Différences
Ci-dessous, les différences entre deux révisions de la page.
Les deux révisions précédentes Révision précédente Prochaine révision | Révision précédente | ||
tuto:monitoring:graylog_to_crowdsec [05/03/2021 15:20] dani [g2cs] |
tuto:monitoring:graylog_to_crowdsec [05/03/2021 19:19] (Version actuelle) rv [Background] |
||
---|---|---|---|
Ligne 3: | Ligne 3: | ||
===== Background ===== | ===== Background ===== | ||
- | Crowdsec' | + | Crowdsec' |
* I really like the idea behind the Journal (systemd-journald), | * I really like the idea behind the Journal (systemd-journald), | ||
* Let's assume we have 40 VM on which we'd like crowdsec agent running. This means something like 40x80MB = 3.2GB of RAM, just for crowdsec | * Let's assume we have 40 VM on which we'd like crowdsec agent running. This means something like 40x80MB = 3.2GB of RAM, just for crowdsec | ||
Ligne 12: | Ligne 12: | ||
===== Send logs from Graylog to ? ===== | ===== Send logs from Graylog to ? ===== | ||
- | As I already have all my logs in Graylog, it'd be better to send this stream of logs to a single crowdsec installation. But, for now, crowdsec doesn' | + | As I already have all my logs in Graylog, it'd be better to send this stream of logs to a single crowdsec installation. But, for now, crowdsec doesn' |
+ | Here's the global flow | ||
+ | |||
+ | {{ : | ||
===== g2cs ===== | ===== g2cs ===== | ||
Ligne 28: | Ligne 31: | ||
* error.log | * error.log | ||
* httpd | * httpd | ||
- | * access | + | * access.log |
+ | * error.log | ||
+ | * zimbra | ||
+ | * mailbox.log | ||
+ | * maxlines is the number of lines each file will get before being truncated. As those log files are only to feed crowdsec, the g2cs daemon will truncate them if they reach this number of lines, so they do not grow indefinitely | ||
+ | |||
+ | <note tip>You can choose a directory on a tmpfs filesystem to improve performance, | ||
+ | |||
+ | < | ||
+ | ===== Configure crowdsec ===== | ||
+ | Now that we have our g2cs daemon running, you can configure crowdsec acquisition to read these files. Something like | ||
+ | <code yaml> | ||
+ | --- | ||
+ | filenames: | ||
+ | - / | ||
+ | labels: | ||
+ | type: syslog | ||
+ | |||
+ | --- | ||
+ | filenames: | ||
+ | - / | ||
+ | - / | ||
+ | labels: | ||
+ | type: nginx | ||
+ | |||
+ | --- | ||
+ | filenames: | ||
+ | - / | ||
+ | - / | ||
+ | labels: | ||
+ | type: apache2 | ||
+ | |||
+ | --- | ||
+ | filenames: | ||
+ | - / | ||
+ | labels: | ||
+ | type: zimbra | ||
+ | </ | ||
+ | |||
+ | ===== Install the syslog output plugin on Graylog ===== | ||
+ | OK, now that we have crowdsec and g2cs ready, we need to send our logs from Graylog to g2cs. For this, we'll use the [[https:// | ||
+ | |||
+ | ===== Create a syslog output ===== | ||
+ | Now in Graylog, you can create a new output. Go in System -> Outputs. Select the " | ||
+ | |||
+ | {{ : | ||
+ | |||
+ | Now, configure your Syslog output like this : | ||
+ | |||
+ | {{ : | ||
+ | |||
+ | * Choose the UDP protocol | ||
+ | * Enter the DNS name or IP address of your server running g2cs | ||
+ | * Choose the port on which g2cs bind | ||
+ | * Choose the CEF message format | ||
+ | |||
+ | ===== Assign output to streams ===== | ||
+ | Now, you can assign in Graylog your new output to the streams you want. Go in the Stream menu, then, " | ||
+ | |||
+ | {{ : | ||
+ | |||
+ | And assign your Syslog output | ||
+ | |||
+ | {{ : | ||
+ | |||
+ | You should now see logs flowing from Graylog, to crowdsec. I'm using this on a small graylog setup, ingesting about 400msg/sec, out of which ~200msg/sec are parsed by my single crowdsec install. I just have to install the bouncers where I want to react to all the agressive IP collected on all my servers | ||
+ | |||
+ | ===== Run crowdsec with less privileges ===== | ||
+ | Bonus point : as crowdsec only access logs from the g2cs daemon, you can run both as a less privileged user, instead of root. First, create an unprivileged user | ||
+ | |||
+ | <code bash> | ||
+ | useradd -r -s / | ||
+ | </ | ||
+ | |||
+ | Now adapt the systemd unit for crowdsec, eg in / | ||
+ | |||
+ | <code ini> | ||
+ | Service] | ||
+ | User=g2cs | ||
+ | Group=g2cs | ||
+ | </ | ||
+ | |||
+ | And create a systemd unit for g2cs itself, / | ||
+ | |||
+ | <code ini> | ||
+ | [Unit] | ||
+ | Description=Graylog to Crowdsec syslog daemon | ||
+ | After=syslog.target | ||
+ | |||
+ | [Service] | ||
+ | Type=simple | ||
+ | ExecStart=/ | ||
+ | User=g2cs | ||
+ | Group=g2cs | ||
+ | Restart=always | ||
+ | PrivateTmp=yes | ||
+ | PrivateDevices=yes | ||
+ | ProtectSystem=full | ||
+ | ProtectHome=yes | ||
+ | NoNewPrivileges=yes | ||
+ | SyslogIdentifier=g2cs | ||
+ | |||
+ | # Allow binding on privileged ports | ||
+ | CapabilityBoundingSet=CAP_NET_BIND_SERVICE | ||
+ | AmbientCapabilities=CAP_NET_BIND_SERVICE | ||
+ | |||
+ | |||
+ | [Install] | ||
+ | WantedBy=multi-user.target | ||
+ | </ |