Wazuh and Suricata on Turris

Linux IDS Networking

The Turris router is a very interesting linux-based networking device with plenty of computing power, it would be pity to use it only as a regular OpenWRT router. It today's world of cyber attacks it can be used as a network monitoring device with an IDS (Intrusion detection system) capability.

How does it work?

The IDS is a system that monitors network traffic for malicious activity like botnets talking to command servers and policy violations (e.g. using torrents in company network). There are multiple implementations of NIDS (Network intrusion detection system), e.g. very popular Snort, Zeek/Bro,.. The Suricata IMHO the most popular solution today as it's multithreaded and uses a same format of the rules as the Snort.

wazuh

The working principle is simple: Suricata listens on one or multiple network interfaces and analyzes the incoming packets, the packets are matched against a set of rules describing what data we are interested in. If the packet matches any enabled rule a log entry is created (with details like source/destination IP, protocol, rule that was triggered, rule description, severity of the captured packet (info/network breach,...)). Optionally also the raw packet data can be attached to the log for further analysis by a security team. That's not all, the suricata can also log DNS requests, expired TLS ceritificates in the communication, etc.

As the security threats are always evolving, the rules for matching malicious data has to evolve too and therefore we have to periodically fetch the update. The another part of the puzzle is the suricata-update script that pulls set of rules from the internet and assembles a rule set for the suricata. There are dozens of various free and payed rule sets, the most used free set is the Emerging threats, it's usually bundled together with the suricata and it's the default set for the suricata-update. If you check the help for the script, there's a way to add around a dozen of other sources. The suricata-updatecan be run periodically by cron, keeping the rules up to date with the ever evolving world of attacks.

Once you start the Suricata, it will log all the alerts into /var/log/suricata/eve.json and several other logs in the same directory. Nice, but it's annoying to check all the logs directly on the turris all the time, also searching and visualizing the data is not very intuitive and simple. We need some tool to store and analyze these logs. This part is the most complicated one during the initial setups. There are thousands of options for this, e.g. professional tool OSSIM, nice looking EveBox frontend, etc.

Today it's popular to use Elasticsearch to store the logs on the central server and Kibana to generate visualizations. If you are running the whole stack on a single machine, it's quite straightforward to configure tools like filebeat to parse the logs and store them in Elastic. Building custom Kibana dashboards isn't very easy, so it's better to use e.g. the EveBox to see the IDS events.

But if you run elasticsearch server on a different machine, it's a completely different story. There's no filebeat release for Turris, therefore you can't simply send logs to elastic in the ordinary way. I had some minor luck with sending the logs over syslog-ng to remote machine, but it was rather complicated to get them into the elastic in format I needed for EveBox. Here steps in the Wazuh SIEM (Security Information and Event Management) solution that's based on elasticsearch and has a suricata events support. It's an overkill as hell, as it's designed to perform analysis of security events, rootkit detection, log analysis, etc. on many servers setup, but I wanted to deploy such solution anyway one day, so why not to do it now.

Install Suricata IDS on Turris

On Turris, run:

# Install suricata package and suricata-update script
opkg install python3-pip suricata-bin
pip3 install suricata-update

# fetch suricata rules
suricata-update -D /etc/suricata

Edit the suricata configuration in /etc/suricata/suricata.yaml, make sure the eve log is enabled, I would advise to disable all the other logs, as we won't need them and they tend to grow really fast, also disable all sub-logs (DNS, HTTP,...) in the eve log if you don't need them. Also check the log rotate options to avoid logs growing too big. Finally, configure the suricata to use rules generated by suricata-update:

default-rule-path: /etc/suricata/rules
rule-files:
  - suricata.rules

The following snippet can be used to fetch the suricata rules periodically, save it to /etc/cron.d/suricata_rules, it will run every day at 05:04:

5 4  * * * root /etc/init.d/suricata stop; /usr/bin/suricata-update -D /etc/suricata; /etc/init.d/suricata start

Unfortunately, the suricata package in the opkg repo doesn't contain the init file, so we have to create it ourselves, e.g. in /etc/init.d/suricata:

#!/bin/sh /etc/rc.common

START=99
PIDFILE=/var/run/suricata.pid

start() {
        mkdir -p /var/log/suricata
        suricata -c /etc/suricata/suricata.yaml -i br-lan -D --pidfile $PIDFILE
}

stop() {
        kill -SIGKILL `cat $PIDFILE`
        rm $PIDFILE
}

Now you can start suricata as any other service by running /etc/init.d/suricata start. To test if it works as expected, run on computer connected to Turris LAN interface curl -A "BlackSun" google.com or curl testmyids.com and check the /var/log/suricata/eve.log, the event should be logged there.

The initial rule set can be quite annoying as it generates many false positives and info messages, to disable them, you can create /etc/suricata/disable.conffile accoring to the docs.

Wazuh

I'll skip the Wazuh manager installation as it's well documented. Once you get the manager running, you can add agent on the Turris and enjoy, right? Not so fast, there's no wazuh-agent package for Turris, there's arm support, but we'll need to compile it ourselves, either on turris or cross-compile it on another machine. As I had a spare Turris lying on my table, I decided to build it directly on Turris instead of fighting with crosscompilers.

First, we need to install several packages and create some lib files that are missing by default.

opkg install perl perlbase-findbin perlbase-pod perlbase-storable perlbase-feature libpthread librt coreutils-install
ar -rc /usr/lib/libdl.a
ar -rc /usr/lib/libl.a
ar -rc /usr/lib/libpthread.a
ar -rc /usr/lib/librt.a

Now download and unpack the wazuh source files:

curl -Ls https://github.com/wazuh/wazuh/archive/v4.1.5.tar.gz | tar zx
cd wazuh-4.1.5/

Now you can start the build process, however it will fail shortly due to some missing headers,... We'll repair this afterwards. Make sure you set install path to /opt/ossec instead of default /var/ossec, the /var mount point is not for persistent files on the Turris.

./install

Once it fails:

  • add <linux/limits.h> to headers in src/external/procps/readproc.c
  • remove line pthread_rwlockattr_setkind_np(&attr, PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP); from src/shared/log_builder.c and src/logcollector/logcollector.c
  • edit src/rootcheck/os_strings.c and change #elif defined Darwing... at the beginning to #else and remove the original else branch

Run the install again, now it should compile and install correctly.

Finally create the init script for the wazuh-agent in /etc/init.d/wazuh-agent:

#!/bin/sh /etc/rc.common

START=99

start() {
        /opt/ossec/bin/ossec-control start
}

stop() {
        /opt/ossec/bin/ossec-control stop
}

Now you can register the agent to wazuh server and check if it appears in the agents dashboard.

Installing prebuilt agent on Turris

If you cross-compile or build on a different Turris than the target one, you can use the files created to install wazuh-agent without building again. Simply copy the whole wazuh folder to the target server, install coreutils-install package, edit etc/preloaded-vars to install only files in bin folder (option down below in the initial section of the file) and run install script

Listening to Suricata data

Edit /opt/ossec/etc/ossec.conf and restart wazuh-agent service:

...
  <localfile>
    <log_format>json</log_format>
    <location>/var/log/suricata/eve.json</location>
  </localfile>
...

Try to trigger the suricata rule as mentioned before, the corresponding event should appear both on the eve.json log and in the Wazuh Security events dashboard.

Next Post