Home IDS with Snort and Snorby

Kevin Breen -
This post is old. If you want to deploy an IDS at home I suggest looking at https://securityonion.net/ which provides an open source Security Appliance with Snort and many other features.

An Intrusion Detection System at is simplest is a network monitoring tool. It is designed to match patterns in network traffic that can be used to indicate malware infections, bad traffic or policy violations. You can read more about IDS here. I have discussed IDS installation in one of my early posts. In that post I used a pre-built, ready to deploy IDS solution. In this series i am going to install and configure each element individually to give me a better understanding of how the technologies work and interact with each other.

This simplified diagram shows my network and IDS placement.

I have Two sensors monitoring network traffic at two seperate points. One protecting my home network the other connected to my malware lab which helps me write rules and identify malware based on its traffic. Both sensors report in to a single management console which means I can monitor both from a single interface.

Our main appliance will house the following

  • Snort - This is the sensor component its responsible for monitoring the raw traffic and comparing the traffic to rules.
  • PullledPork - This is our rule management application.
  • Barnyard2 - This processes the alerts generated by snort and processes them in to a database format.
  • Snorby - This is the visual front end to the event data that is written in to the database.

_For any additional sensors we can skip the Snorby install. _

Installation

I'm building this in a Virtual Ubuntu 14.04-x86_64 server. It has two cores and 2 Gb RAM, which should be enough for my small home network. It has two Network Interfaces one for management the other will run promiscuous mode and is connected to the spanned port.

First install the base OS I am not going to cover this in any detail except to say I set the following details at the relevant points.

  • Set hostname to snort-ids.
  • User - sensor
  • Dont Encrypt Home Folder
  • Select ssh-server from the optional install page

Once the base OS is installed, with the wonder that is snapshotting, I take a snapshot so when if I mess things up I can revert back to a clean state.

Once the install has finished and you're logged in as the user jump up to a root prompt with:

sudo su

Update the host with:

apt-get update && apt-get upgrade

Configure our interfaces.

nano /etc/network/interfaces

add or edit the following lines to match your network

# The primary network interface
auto eth0
iface eth0 inet static
    address 192.168.1.200
    netmask 255.255.255.0
    gateway 192.168.1.1
    dns-nameservers 8.8.8.8 192.168.1.1

# The monitor Interface
auto eth1
iface eth1 inet manual
    up ifconfig eth1 up promisc
    down ifconfig eth1 down -promisc</pre>

Reset the interfaces to their new config with the following commands.

ifdown eth0
ifup eth0
ifup eth1</pre>

Check both interfaces came up and eth1 lists PROMISC.

before we go any further lets make sure we are correctly capturing traffic on the interface. If we are not it will cause us issues with testing further down the line. Plus our IDS will be pointless without capture traffic :)

tcpdump -i ens192 port 80

You should see traffic scrolling the page. If you don't check your network tap and the interface configuration.Hit Ctrl+C to stop. We filter on port 80 to make sure we are not just getting broadcast traffic and that promiscuous mode is working as intended.

Dependencies

Lets grab the dependencies we need for the install

apt-get -y install build-essential libtool automake gcc flex bison libnet1 libnet1-dev libpcre3 libpcre3-dev autoconf libcrypt-ssleay-perl libwww-perl git zlib1g zlib1g-dev libssl-dev mysql-server libmysqlclient-dev apache2 imagemagick wkhtmltopdf ruby1.9.3 libyaml-dev libxml2-dev libxslt1-dev openssl libreadline6-dev unzip libcurl4-openssl-dev apache2-threaded-dev libapr1-dev libaprutil1-dev

there are a couple of items that we want to install manually to stay on the latest version.

mkdir tmp_build
cd tmp_build
wget https://libdnet.googlecode.com/files/libdnet-1.12.tgz
tar zxf libdnet-1.12.tgz
cd libdnet-1.12
./configure
make && make install
cd ..
wget http://www.tcpdump.org/release/libpcap-1.6.2.tar.gz
tar zxf libpcap-1.6.2.tar.gz
cd libpcap-1.6.2
./configure
make && make install
ldconfig

Snort - Install

We are going to download and compile snort based on the lastest stable release from snort.org

cd ..
wget https://snort.org/downloads/snort/snort-2.9.7.0.tar.gz
wget https://snort.org/downloads/snort/daq-2.0.4.tar.gz

Install Data Acquisition Library

tar zxf daq-2.0.4.tar.gz
cd daq-2.0.4
./configure
make && make install
ldconfig

Install Snort Engine

cd ..
tar zxf snort-2.9.7.0.tar.gz
cd snort-2.9.7.0
./configure --enable-sourcefire
make && make install
ln -s /usr/local/lib/libdnet.1.0.1 /usr/lib/libdnet.1<

Lets test snort works before we go any futher. Dont worry too much about the output for now, we are just testing the binaries launch correctly and without error.

snort

Some housekeeping for snort

groupadd snort 
useradd snort -d /var/log/snort -s /sbin/nologin -c SNORT_IDS -g snort
mkdir /var/log/snort
chown snort:snort /var/log/snort
mkdir /etc/snort
mv etc/* /etc/snort/

Snort - Rules

Snort needs rules. you can write your own but there are also community contributed rules and Snort provide their own rules. Subscriber rules are the latest rules but require a paid subscription. Registered rules are free and typically one month behind the subscribers rules. Register on https://snort.org to get your oinkcode.

To make life easier for us in the future we are going to set up pulled pork to automagically download and process all the rules we need.

cd ..
wget https://pulledpork.googlecode.com/files/pulledpork-0.7.0.tar.gz
tar zxf pulledpork-0.7.0.tar.gz
cd pulledpork-0.7.0
cp pulledpork.pl /usr/sbin/
chmod 755 /usr/sbin/pulledpork.pl
cp etc/* /etc/snort/
cpan install LWP::Protocol::https
cpan install Crypt::SSLeay 
cpan Mozilla::CA IO::Socket::SSL

If your running CPAN for the first time follow the onscreen instructions for initial setup. You can safely accept all the default values.

At the time of writing pulled pork didn't like grabbing items over SSL, to fix this we set pulled pork to not validate ssl certificates. If pulledpork generates an error 500 you might want to try this as well.

nano /usr/sbin/pulledpork.pl

After Vars here and before _# we are gonna need these _add the following line.

$ENV{'PERL_LWP_SSL_VERIFY_HOSTNAME'} = 0;$ENV{'PERL_LWP_SSL_VERIFY_HOSTNAME'} = 0;

Edit the pulled pork conf file

nano /etc/snort/pulledpork.conf

  • anywhere you see  replace it with your code
  • rule_path=/etc/snort/rules/snort.rules
  • local_rules=/etc/snort/rules/local.rules
  • sid_msg=/etc/snort/sid-msg.map
  • snort_path=/usr/local/bin/snort
  • config_path=/etc/snort/snort.conf
  • distro= Ubuntu-10-4
  • black_list=/etc/snort/rules/black_list.rules
  • snort_control=/usr/bin/snort_control

Comment out #IPRVersion=/usr/local/etc/snort/rules/iplists

Uncomment rule_url=https://rules.emergingthreatspro.com/|emerging.rules.tar.gz|open

Before we can pull down the rules we need to configure snort.

nano /etc/snort/snort.conf

There is a lot in here so i would advise reading through and setting up what you need. The edits required for us are as follows

  • ipvar HOME_NET 192.168.1.0/24
  • ipvar EXTERNAL_NET !$HOME_NET

If you run multiple lan segments and your DNS, SQL, WEB servers etc sit outside of your defined HOME_NET we can set them here. Otherwise leave them as $HOME_NET

If you run Web Proxy services you will need to check your proxy port is set under portvar HTTP_PORTS additionaly if your running standard services on non standard ports you should modify portvars as required.

set rule paths to be absolute

  • var RULE_PATH /etc/snort/rules
  • var SO_RULE_PATH /etc/snort/rules/so_rules
  • var PREPROC_RULE_PATH /etc/snort/rules/preproc_rules
  • var WHITE_LIST_PATH /etc/snort/rules
  • var BLACK_LIST_PATH /etc/snort/rules

Look for the section that lists # site specific rules.

Remove all the include $RULE_PATH/.... adding the following two lines.

  • include $RULE_PATH/local.rules
  • include $RULE_PATH/snort.rules

The final edit we need to set the output format for snort alerts under Step #6 add the following line

  • output unified2: filename snort.log, limit 128

save and close the file

we need to set some paths and permissions.

cd /etc/snort
chown -R snort:snort *
mkdir -p /usr/local/lib/snort_dynamicrules
mkdir /etc/snort/rules
touch /etc/snort/rules/so_rules.rules
touch /etc/snort/rules/local.rules
touch /etc/snort/rules/white_list.rules

With all the config files set lets pull down some rules.

pulledpork.pl -c /etc/snort/pulledpork.conf

If pulled pork runs without errors we can set a cron job so it will run every day. Snort needs to be restarted in order to apply new rules. If you prefer to do this manually to avoid possible errors don't add the second cron job.

nano /etc/crontab

  • 0 0 * * * root /usr/sbin/pulledpork.pl -c /etc/snort/pulledpork.conf
  • 0 15 * * * root service snort restart

We can sniff traffic, we have rules and we are configured lets run a quick test.

snort -T -i ens192 -u snort -g snort -c /etc/snort/snort.conf

This tells snort to run in test mode with user and group against our interface and use our config file.

Give it a minute and you should finally see

Snort successfully validated the configuration! Snort exiting

If you get errors you will need to fix these before continuing.

Now lets try a real test. We are going to create a simple rule that we can trigger at any time. This gives us an effective method of testing snort without having to worry about waiting for something to fire. The rule is just designed to match on any icmp traffic which means a ping anywhere being monitored should alert.

nano /etc/snort/rules/local.rules

add the following line. We will discuss rules in more detail later.

alert icmp any any -> any any (msg: "ICMP Packet found"; classtype:bad-unknown; sid:500000; rev:1;)

Now run a live test with

snort -i eth1 -u snort -g snort -c /etc/snort/snort.conf

once you see the output "Commencing packet processing" trigger our test rule by pinging a device that's on the monitored network, then hit Ctrl+C to stop snort. Scroll back through the output and look for the sections that lists Action Stats:

Action Stats:  
 Alerts: 8 ( 0.915%)
 Logged: 8 ( 0.915%)
 Passed: 0 ( 0.000%)

You should see a count greater than 0 for Alerts and Logged. If you see 0 check your rule and that your actually capturing traffic on the interface.

Now we have tested our sensor lets get it running at boot time before we make any more adjustments.

Snort - Boot

nano /etc/init/snort.conf

add the following in to the file

description "Snort Service"
start on runlevel [2345]
stop on runlevel [!2345]
script
exec snort -q -i eth1 -u snort -g snort -c /etc/snort/snort.conf -D
end script

Save and exit

Set and start the service with

service snort start```

Once started check its still running

```# ps -A | grep snort
root@snort-ids:/home/sensor/tmp_build/snort-2.9.7.0# ps -A | grep snort
 4586 ? 00:00:00 snort```

Great we have a functional IDS but its not what you would call intuitive to understand what alerts are being triggered. For this we need to install a front end. This comes in two parts. Barnyard will read the output from snort and store it in to a SQL database. From there we can use a variety of tools to read the database. This guide will be using Snorby.

### Barnyard2 - Install

First we need to configure the SQL database to store the alerts.

Secure the SQL install with

```mysql_secure_installation```

If you set a strong root sql password earlier you can leave the root password as is is. Answer yes to every other question.

grab the SQL Schema

cd /home/sensor/tmp_build/
wget https://raw.githubusercontent.com/firnsy/barnyard2/master/schemas/create_mysql


Create users and tables to access the DB

```sql
mysql -u root -p
create database snort;
grant all on snort.* to snort@'localhost' IDENTIFIED BY 'set a strong password here';
flush privileges; 
use snort; 
source create_mysql; 
show tables;

The output of show tables should produce the following

 +------------------+
 | Tables_in_snort  |
 +------------------+
 | data             |
 | detail           |
 | encoding         |
 | event            |
 | icmphdr          |
 | iphdr            |
 | opt              |
 | reference        |
 | reference_system |
 | schema           |
 | sensor           |
 | sig_class        |
 | sig_reference    |
 | signature        |
 | tcphdr           |
 | udphdr           |
 +------------------+
 16 rows in set (0.00 sec)
> quit

With SQL working we need to install barnyard.

wget https://github.com/firnsy/barnyard2/archive/v2-1.13.tar.gz
tar zxf v2-1.13.tar.gz
cd barnyard2-2-1.13/
./autogen.sh
./configure --with-mysql --with-mysql-libraries=/usr/lib/x86_64-linux-gnu 
make && make install

with the install complete without errors lets move on to the configuration.

Barnyard2 - Configure

mv /usr/local/etc/barnyard2.conf /etc/snort/
nano /etc/snort/barnyard2.conf

we need to tell barnyard how to connect to our new sql database. At the end of the file add the following line:

  • output database: log,mysql, user=snort password=yourpasshere dbname=snort host=localhost

To uniquely identify this sensor in the database modify theses two lines as appropriate

  • config hostname snort-ids
  • config interface eth1

Set some file paths.

mkdir /var/log/barnyard2
chown snort.snort /var/log/barnyard2
touch /var/log/snort/barnyard2.waldo
chown snort.snort /var/log/snort/barnyard2.waldo

Once all the changes have been made test barnyard runs correctly with

barnyard2 -c /etc/snort/barnyard2.conf -d /var/log/snort -f snort.log -w /var/log/snort/barnyard2.waldo -g snort -u snort

On its first run Barnyard processes a lot of background tasks and this can take several minutes to complete if you are running with a lot of rules. wait for the screen to display

-== Initialization Complete ==-

Once you see this line appear stop barnyard with _Ctrl+C and _check the sql database to make sure events are being stored.

mysql -u snort -p snort

select count(*) from event;
+----------+
| count(*) |
+----------+
| 28       |
+----------+
1 row in set (0.02 sec)
quit

Barnyard2 - Boot

nano /etc/init/barnyard2.conf

add the following in to the file

description "Barnyard2 Alerts"

start on runlevel [2345]
stop on runlevel [!2345]

script
exec barnyard2 -c /etc/snort/barnyard2.conf -d /var/log/snort -f snort.log -w /var/log/snort/barnyard2.waldo -g snort -u snort -D
end script

Save and exit
Set and start the service with

chmod +x /etc/init/barnyard2.conf && service barnyard2 start
Once started check its still running

ps -A | grep barn
16749 ? 00:00:17 barnyard2

Before we get on to the GUI for managing our alerts let's check that our services start correctly. Reboot the machine and when it comes back up check that snort and barnyard are still running.

ps -A | egrep "snort|barn"
  792 ? 00:05:10 barnyard2
  15026 ? 00:00:05 snort

Snorby - Dependencies

The final part of our installation is a web GUI that we can use to monitor and manage any alerts generated by snort. There are several options, my preference is Snorby.

Jump up to a root prompt with sudo su

Snorby - Setup

Snorby has its own database schema you can set this alongside the snort tables we made for testing earlier or you can drop the old database and use the new one. You cannot add snorby to the existing database. We will use the same account we created for snort so we dont have unused accounts laying around.

mysql -u root -p
> create database snorby;
> grant all on snorby.* to snort@localhost identified by 'snortsqlpassword';
> flush privileges;
> quit

Grab snorby and save it in to the web dir.

cd /home/sensor/tmp_build 
wget -O snorby.zip --no-check-certificate https://github.com/Snorby/snorby/archive/master.zip
unzip snorby.zip 
mkdir /var/www/html/snorby
mv snorby-master/* /var/www/html/snorby

Now we need to edit a couple of files

cd /var/www/html/snorby
nano Gemfile

make the following changes

  • gem 'rake', '0.9.2' to  gem 'rake', '> 0.9.2'
  • after gem 'json', '~> 1.7' add gem 'thin'

under the group(:development) section comment out the gem thin line as show below.

group(:development) do
     gem "letter_opener"
# gem 'thin'

nano Gemfile.lock

change the line rake (0.9.2) to rake (0.9.2.2)

Install the bundles and set the config files

gem install rails bundler passenger
bundle install
cp config/snorby_config.yml.example config/snorby_config.yml
cp config/database.yml.example config/database.yml

Edit the config file

nano config/snorby_config.yml

under production modify the rules section to match. This will allow Snorby to display the full snort rule that triggered the event.

rules:
- "/etc/snort/rules"

You can edit other sections as required, it will still work leaving them as default values.

Edit the database config file

nano config/database.yml

set the username and password to the values we created when we configured the snorby database

Run With Apache2

passenger-install-apache2-module

follow the onscreen instructions

once the install has finished scroll through the output and copy the lines similar to those below we will need them.

LoadModule passenger_module /var/lib/gems/1.9.1/gems/passenger-4.0.58/buildou$
 
 PassengerRoot /var/lib/gems/1.9.1/gems/passenger-4.0.58
 PassengerDefaultRuby /usr/bin/ruby1.9.1

nano /etc/apache2/sites-availiable/snorby.confs

Paste the code we copied above and then add the code below making sure to change the Servername to match your own IP address or hostname

 Servername 192.168.1.200
 DocumentRoot /var/www/html/snorby/public
 
 AllowOverride all
 Order allow,deny
 Allow from all
 Options -MultiViews

Enable the site with

ln -s /etc/apache2/sites-available/snorby.conf /etc/apache2/sites-enabled/snorby.conf
rm /etc/apache2/sites-enabled/000-default.conf

Set file permissions on the Snorby dir

chown -R www-data:www-data /var/www/html/snorby

The final thing we need to change is to modify our barnyard install to write events in to Snorby's database. As mentioned earlier you can modify the existing line or you can use both, I Modify the existing line.

nano /etc/snort/barnyard2.conf

edit the output database line to match Snorby's database

  • output database: log,mysql, user=snort password=yourpasswordhere dbname=snorby host=localhost

Stop barnyard if it is already running.

service barnyard stop

run barnyard from the command line to check the new database settings applied without error. This can take several minutes as it did previously depending on how many rules are enabled.

barnyard2 -c /etc/snort/barnyard2.conf -d /var/log/snort -f snort.log -w /var/log/snort/barnyard2.waldo -g snort -u snort

As before once you see barnyard has successfully initialized stop with Ctrl+C.

Then start Snorby.

RAILS_ENV=production bundle exec rake snorby:setup

Snorby will now build all its tables and start the Snorby Workers, after Snorby has started restart your machine a final time. Once the OS has booted point your favorite browser at http://your.ip.

The first load may take a moment once you get the login screen use the default credentials snorby@snorby.org | snorby.

If you see a warning about workers not currently running

Administration -> Worker & Job Que

Worker Options -> Start Worker

After a minute you should see:

Replaced with:

If this still fails then check the owner of /var/www/html/snorby/pid it should be owned by www-data if not then re run the chown command from earlier.

Returning to the dash board you should start to see events populating the dashboard.

If you don't see events try to trigger our test rule by pinging a device that's on the monitored network. If everything works then remove our test rule by deleting the line in /etc/snort/rules/local.rules. Or placing a # at the start of the line.

That's pretty much it, add yourself a new administrator account to Snorby and check you can log in with this account before removing the default account. In future posts we will look at creating our own rules and tuning rules for our network.

This process builds our primary sensor and management. Adding additional sensors is as simple as repeating these steps for each device. When you reach the section on configuring Barnyard2 instead of writing to a local mysql database, point it to the mysql database on the primary sensor. Make sure you set a unique hostname to identify it as a different sensor and you should be good to go. You dont need to run the Snorby install or configuration.