Skip to main content

CrowdSec Security Engine Setup Health-Check

Welcome to the interactive health check of your CrowdSec setup. We'll have a Top-Down approach:
First, check the proper functioning of the Security Engine's finality:

  • 📡 Detecting behaviors on your services.
  • 🔗 Connectivity with CrowdSec network to retrieve threats lists.
  • 🛡️ Protecting your services by remediating alerts automatically with bouncers.

If the initial checks are not conclusive, we'll dive into a comprehensive troubleshooting for each section.

📡 Detection checks​

Trigger CrowdSec's test scenarios​

info

Check that your Security Engine properly reads and parses the logs of the services you protect.
The HTTP collection and the Linux collection contain dummy scenarios allowing you to prove your Security Engine works as intended, without having to do penetration tests, risking getting you banned from your own server.

Details

Test HTTP detection Let's trigger the crowdsecurity/http-generic-test dummy scenario by calling a probe path on your web server.

1️⃣ Request your service URL with the following path: /crowdsec-test-NtktlJHV4TfBSK3wvlhiOBnl 

curl -I https://<your-service-url>/crowdsec-test-NtktlJHV4TfBSK3wvlhiOBnl

2️⃣ You should see an alert for the scenario crowdsecurity/http-generic-test

sudo cscli alerts list -s crowdsecurity/http-generic-test

Note that this scenario has a delay of 5 minutes before it can re-trigger (blackhole parameter of the scenario).

Details

Test SSH detection Let's trigger the crowdsecurity/ssh-generic-test dummy scenario by logging in to your server via SSH with a specific user.

1️⃣ Try to authenticate to your server via SSH using the following user: crowdsec-test-NtktlJHV4TfBSK3wvlhiOBnl. 

ssh crowdsec-test-NtktlJHV4TfBSK3wvlhiOBnl@<your-ip>

2️⃣ You should see an alert for the scenario crowdsecurity/ssh-generic-test

sudo cscli alerts list -s crowdsecurity/ssh-generic-test

Note that this scenario has a delay of 5 minute before it can re-trigger (blackhole parameter of the scenario).

Details

Test CrowdSec WAF with an AppSec detection If you are using an AppSec-capable bouncer and have configured CrowdSec WAF, you can test the detection of an AppSec scenario. It's similar to the HTTP detection test, but it will trigger crowdsecurity/appsec-generic-test.

Here is how to trigger the crowdsecurity/appsec-generic-test dummy scenario by calling a probe path on your web server.

1️⃣ Request your service URL with the following path: /crowdsec-test-NtktlJHV4TfBSK3wvlhiOBnl 

curl -I http://your-service-url/crowdsec-test-NtktlJHV4TfBSK3wvlhiOBnl

2️⃣ You should see an alert for the scenario crowdsecurity/appsec-generic-test

sudo cscli alerts list -s crowdsecurity/appsec-generic-test

Note that this scenario has a delay of 1 minute before it can re-trigger (blackhole parameter of the scenario).

Were all the tests successful ?​

Were all the tests related to your setup successful ? If so, you can proceed to the next phase of the health check: Connectivity checks.

If not, check the troubleshooting section below

🚨 Detection Troubleshooting

No alerts triggered? Let's investigate: Here are some tests to identify where the issue might be.

If you installed CrowdSec Security Engine on the same Host as the service you want to protect, the install wizards should have automatically detected the service and installed the appropriate parsers and scenarios.

If you have non-default paths or format for your logs, or if you chose other installation methods (docker, kubernetes..), you may need to manually install the parsers and scenarios.

This troubleshooting section will help you identify the issue and guide you through the necessary steps to fix it.

Acquisition metrics check: are your logs read and parsed?

The acquisition and parsing aspect of CrowdSec is crucial, as it tells The Security Engine which logs to read and how to parse them. You can setup multiple datasources (files, syslog, etc.), for more details you can refer to the datasources documentation.

Let's do a Top Down check using the cscli metrics command to see if your logs are being read and parsed correctly.

sudo cscli metrics show acquisition parsers
  • You should see the names of the log files/stream configured in the acquisition files, and the number of lines parsed for each of them.
  • The number of "Lines parsed" should be non-zero for each of the files you configured in the acquisition section.
  • The parsers metrics show you what parsers were successfully used. Look for the name of the parsers installed for the logs you're reading

🚨 If this command fails, check the CrowdSec Service Troubleshooting section

🚨 If you don't see the log names supposed to be parsed, check the Acquisition Troubleshooting section

🚨 If you don't see parsed lines, check the Collection Troubleshooting section**

Acquisition Troubleshooting: are your logs properly declared as datasources

Your acquisition configuration is located in the crowdsec directory under acquis.yaml or acquis.d/anyName.yaml.
The crowdsec directory changes depending on your OS and installation method, for example, on Debian like OS it is usually located in /etc/crowdsec/.

You may want to check your current acquisition configuration or add a new acquisition file.

  • Check that the datasource is properly setup: file path or correct port for syslog server...
  • Ensure that the type declared matches the parser expected to be used: nginx, apache, syslog, etc.
  • for more information, check the documentation for the various datasources
Collection Troubleshooting: are the parsers and scenarios installed?

Collection installation​

  • On host installations your existing service should have been detected and appropriate collections (containing parsers and scenarios) installed automatically.
  • How ever on other kind of installations, or in case of failed detection, you may need to install the collections manually.

Check the installed collections with the command: 

sudo cscli collections list
You can also list individual parsers and scenarios with: 
sudo cscli parsers list
sudo cscli scenarios list

  • If you see the parsers and scenarios related to your service, then you're good to go.

  • To install new collection, check the CrowdSec Hub for a collection named after your service (e.g., nginx, apache, linux, etc.).

  • If you find the collection, install it using the command shown on the hub page and follow the instructions provided for the acquisition file.

Log format mismatch​

  • If your logs are in a custom format, you may need to create your own parsers or modify the existing ones to match your log format.
  • You can find more information on how to create your own parsers in the CrowdSec documentation.
CrowdSec Service Troubleshooting: is the CrowdSec service running?

CrowdSec Connectivity checks​

Check that your Security Engine can communicate with the CrowdSec Central API and receive the community blocklist and additional ones.

// Do we make them check console enroll here? or just cscli capi status? - Recommend console enrollment here; the previous checks establish core functionality. Enrollment becomes the next logical step.

sudo cscli capi status
  • Should show: "You can successfully interact with Central API (CAPI)"
  • If this test fail, try to enrol to the console to check what's going on.
Network Connectivity Troubleshooting
  • online api credentials exist in /etc/crowdsec/config/online_api_credentials.yaml (default path)
  • Firewall rules blocking outbound connections to the CrowdSec Central API.
  • DNS resolution issues.
  • Proxy server configuration.
  • Connectivity issues within Docker containers.

Remediation checks​

  • Canary tester

(This section requires a robust, safe, and automated canary tester solution. The biggest challenge is preventing users from accidentally banning themselves. This section can be skipped on V0, as you mentioned, and focused on later. For V1, It could be a URL to a third party machine for testing, or a private machine dedicated to running a canary test)

Bouncer Troubleshooting
  • Verifying that the bouncer is registered and active.
  • Checking bouncer logs for errors.
  • Troubleshooting communication issues between the bouncer and the CrowdSec LAPI.
  • Firewall configuration for the bouncer.
  • Specific troubleshooting steps for common bouncers (firewall-bouncer, nginx-bouncer, etc.).)
Profile Troubleshooting: Steps
  • Verifying profile configuration.
  • Troubleshooting errors related to profile application.
  • Understanding the impact of profiles on detection and remediation.)
CrowdSec ConsoleCrowdSec Console