Monitoring Netflow with NFsen

Introduction

Goals

Notes

Configure Your Collector

We will collect netflow data on srv1-gY.lab.workalaya.net server.

Please find your classmates that are using the same group as you. Get in to a group and do the following exercise together. That is, pick one person who will configure your group's srv1-gY server using ansible.

Optionally you can configure own VM as netflow collector as well.

This document follows installing on vmX-gY.lab.workalaya.net instead of using srv1-gY.lab.workalaya.net.

Install NFDump and associated software

NFdump is part of the Netflow flow collector tools, which includes:

nfcapd, nfdump, nfexpire, nfprofile, nfreplay, nftrack, nfanon

Create a new ansible playbook named nfdump-nfsen.yml to install NFDump.

(venv) vmX-gY@ansible-host:~/ansible-playbook$ vi nfdump-nfsen.yml

- hosts: nfsen_hosts
  become: true
  tasks:
    - name: ensure package cache is up to date
      apt:
        update_cache: yes
        cache_valid_time: 3600
      tags: install-nfdump, build-nfsen

    - name: install nfdump and dependency packages
      package:
        name: "{{ item }}"
        state: present
      with_items:
        - nfdump
        - rrdtool
        - librrds-perl
        - librrdp-perl
        - libmailtools-perl
        - libsocket6-perl
        - php
      tags: install-nfdump

    - name: stop nfdump
      service:
        name: nfdump
        state: stopped
      tags: install-nfdump

    - name: disable nfdump
      service:
        name: nfdump
        enabled: no
      tags: install-nfdump

update inventory/hosts as following

(venv) vmX-gY@ansible-host:~/ansible-playbook$ vi inventory/hosts

[nagios_hosts]
vmX-gY.lab.workalaya.net

[snmp_hosts]
vmX-gY.lab.workalaya.net

[smokeping_hosts]
vmX-gY.lab.workalaya.net

[netdot_hosts]
vmX-gY.lab.workalaya.net

[rancid_hosts]
vmX-gY.lab.workalaya.net

[nfsen_hosts]
vmX-gY.lab.workalaya.net
srv1-gY.lab.workalaya.net

Now run ansible playbook to install nfdump as following.

(venv) vmX-gY@ansible-host:~/ansible-playbook$ ansible-playbook nfdump-nfsen.yml

PLAY [nfsen_hosts] ******************************************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************************
ok: [vmX-gY.lab.workalaya.net]

TASK [ensure package cache is up to date] *******************************************************************************************************************
[WARNING]: Could not find aptitude. Using apt-get instead

changed: [vmX-gY.lab.workalaya.net]

TASK [install nfdump and dependency packages] ***************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net] => (item=nfdump)
ok: [vmX-gY.lab.workalaya.net] => (item=rrdtool)
ok: [vmX-gY.lab.workalaya.net] => (item=librrds-perl)
changed: [vmX-gY.lab.workalaya.net] => (item=librrdp-perl)
changed: [vmX-gY.lab.workalaya.net] => (item=libmailtools-perl)
ok: [vmX-gY.lab.workalaya.net] => (item=libsocket6-perl)
changed: [vmX-gY.lab.workalaya.net] => (item=php)

TASK [stop nfdump] ******************************************************************************************************************************************
ok: [vmX-gY.lab.workalaya.net]

TASK [disable nfdump] ***************************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

PLAY RECAP **************************************************************************************************************************************************
vmX-gY.lab.workalaya.net   : ok=5    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Testing nfcapd and nfdump

Now login back to your VM and perform following task.

lab@vmX-gY:~$ mkdir /tmp/nfcap-test
lab@vmX-gY:~$ nfcapd -E  -p 9001 -l /tmp/nfcap-test

... after a while, a series of flows should be dumped on your screen.

Stop the tool with CTRL+C, then look at the contents of /tmp/nfcap-test

lab@vmX-gY:~$ ls -l /tmp/nfcap-test

You should see one or more files called nfcapd.<YEAR><MON><DAY><HR><MIN>

Process the file(s) with nfdump:

lab@vmX-gY:~$ nfdump -r /tmp/nfcap-test/nfcapd.201Ywwxxyyzz | less
lab@vmX-gY:~$ nfdump -r /tmp/nfcap-test/nfcapd.201Ywwxxyyzz -s srcip/bytes

You should get some useful information smiley

Installing and setting up NfSen

Now login back to ansible control host.

Update ansible playbook named nfdump-nfsen.yml to install NfSen. In this playbook we will perform following tasks:

(venv) vmX-gY@ansible-host:~/ansible-playbook$ vi nfdump-nfsen.yml

- hosts: nfsen_hosts
  become: true
  vars:
    nfsen_sources:
      - "'rtr-g{{ class_group }}' => {'port'=>'9996','col'=>'#0000ff','type'=>'netflow'},"
    nfsen_bufflen: 20000
  tasks:
    - name: ensure package cache is up to date
      apt:
        update_cache: yes
        cache_valid_time: 3600
      tags: install-nfdump, build-nfsen

    - name: install nfdump and dependency packages
      package:
        name: "{{ item }}"
        state: present
      with_items:
        - nfdump
        - rrdtool
        - librrds-perl
        - librrdp-perl
        - libmailtools-perl
        - libsocket6-perl
        - php
      tags: install-nfdump

    - name: stop nfdump
      service:
        name: nfdump
        state: stopped
      tags: install-nfdump

    - name: disable nfdump
      service:
        name: nfdump
        enabled: no
      tags: install-nfdump

    - name: check for nfsen install directory
      stat:
        path: /var/nfsen
      register: nfsen_check
      tags: build-nfsen

    - name: nfsen
      block:
        - name: download nfsen source
          get_url:
            # url: 'https://sourceforge.net/projects/nfsen/files/stable/nfsen-1.3.8/nfsen-1.3.8.tar.gz/download'
            url: 'http://www.lab.workalaya.net/downloads/nfsen/nfsen-1.3.8.tar.gz'
            dest: /usr/local/src
          retries: 5
          delay: 10

        - name: cleanup nfsen source directory
          file:
            path: /usr/local/src/nfsen-1.3.8
            state: absent

        - name: extract nfsen source
          unarchive:
            src: /usr/local/src/nfsen-1.3.8.tar.gz
            dest: /usr/local/src
            remote_src: yes

        - name: download nfsen patch
          get_url:
            # url: 'http://npnog5.chatur.com.np/nmm/netflow/nfsen.patch'
            url: 'http://www.lab.workalaya.net/downloads/nfsen/nfsen.patch'
            dest: /usr/local/src/nfsen-1.3.8
          retries: 5
          delay: 10

        - name: apply patches to nfsen source
          shell: patch -p0 < {{item}}
          args:
            chdir: '/usr/local/src/nfsen-1.3.8'
          with_items:
            - nfsen.patch
          ignore_errors: true

        - name: create nfsen.conf
          template:
            src: templates/nfsen/nfsen.conf
            dest: /usr/local/src/nfsen-1.3.8/etc/nfsen.conf

        - name: create nfsen user
          user:
            name: netflow
            home: /var/nfsen
            groups: www-data
            append: yes
            shell: /bin/false

        - name: run nfsen installer
          shell: perl install.pl ./etc/nfsen.conf </dev/null
          args:
            chdir: '/usr/local/src/nfsen-1.3.8'

        - name: install nfsen.service systemd
          copy:
            src: files/nfsen/nfsen.service
            dest: /etc/systemd/system/

        - name: enable nfsen
          service:
            name: nfsen
            enabled: yes

        - name: start nfsen
          service:
            name: nfsen
            state: started

      when: nfsen_check.stat.isdir is not defined or nfsen_check.stat.isdir == false
      tags: build-nfsen

now create nfsen.conf template file as

(venv) vmX-gY@ansible-host:~/ansible-playbook$ mkdir -p templates/nfsen
(venv) vmX-gY@ansible-host:~/ansible-playbook$ vi templates/nfsen/nfsen.conf

##############################
#
# NfSen master config file
#
# $Id: nfsen-dist.conf 22 2007-11-20 12:27:38Z phaag $
#
# Configuration of NfSen:
# Set all the values to fit your NfSen setup and run the 'install.pl'
# script from the nfsen distribution directory.
#
# The syntax must conform to Perl syntax.
#
##############################
#
# NfSen default layout:
# Any scripts, modules or profiles are installed by default under $BASEDIR.
# However, you may change any of these settings to fit your requested layout.

#
# Required for default layout
$BASEDIR = "/var/nfsen";

#
# Where to install the NfSen binaries
$BINDIR="${BASEDIR}/bin";

#
# Where to install the NfSen Perl modules
$LIBEXECDIR="${BASEDIR}/libexec";

#
# Where to install the config files
$CONFDIR="${BASEDIR}/etc";

#
# NfSen html pages directory:
# All php scripts will be installed here.
# URL: Entry point for nfsen: http://<webserver>/nfsen/nfsen.php
$HTMLDIR    = "/var/www/html/nfsen/";

#
# Where to install the docs
$DOCDIR="${HTMLDIR}/doc";

#
# Var space for NfSen
$VARDIR="${BASEDIR}/var";

# directory for all pid files
# $PIDDIR="$VARDIR/run";
#
# Filter directory
# FILTERDIR="${VARDIR}/filters";
#

# FORMATDIR for custom printing formats
# FORMATDIR="${VARDIR}/fmt";
#

#
# The Profiles stat directory, where all profile information
# RRD DBs and png pictures of the profile are stored
$PROFILESTATDIR="${BASEDIR}/profiles-stat";

#
# The Profiles directory, where all netflow data is stored
$PROFILEDATADIR="${BASEDIR}/profiles-data";

#
# Where go all the backend plugins
$BACKEND_PLUGINDIR="${BASEDIR}/plugins";

#
# Where go all the frontend plugins
$FRONTEND_PLUGINDIR="${HTMLDIR}/plugins";

#
# nfdump tools path
$PREFIX  = '/usr/local/bin';

#
# nfsend communication socket
# $COMMSOCKET = "$PIDDIR/nfsen.comm";

# BASEDIR unrelated vars:
#
# Run nfcapd as this user
# This may be a different or the same uid than your web server.
# Note: This user must be in group $WWWGROUP, otherwise nfcapd
#       is not able to write data files!
$USER    = "netflow";

# user and group of the web server process
# All netflow processing will be done with this user
$WWWUSER  = "www-data";
$WWWGROUP = "www-data";

# Receive buffer size for nfcapd - see man page nfcapd(1)
$BUFFLEN = {{nfsen_bufflen}};

# list of extensions for each collector. See argument -T
# for nfcapd(1) for more detailes.
# defaults to empty -> compatible to nfdump-1.5.8
# $EXTENSIONS = '';
# Example:
# $EXTENSIONS = 'all';
# $EXTENSIONS = '+3,+4';
#
# Directory sub hierarchy layout:
# Possible layouts:
#
# 0 default     no hierachy levels - flat layout - compatible with pre NfSen versions
# 1 %Y/%m/%d    year/month/day
# 2 %Y/%m/%d/%H year/month/day/hour
# 3 %Y/%W/%u    year/week_of_year/day_of_week
# 4 %Y/%W/%u/%H year/week_of_year/day_of_week/hour
# 5 %Y/%j       year/day-of-year
# 6 %Y/%j/%H    year/day-of-year/hour
# 7 %Y-%m-%d    year-month-day
# 8 %Y-%m-%d/%H year-month-day/hour
$SUBDIRLAYOUT = 1;

# Compress flows while collecting 0 or 1
$ZIPcollected    = 1;

# Compress flows in profiles 0 or 1
$ZIPprofiles     = 1;

# Interrupt expire -- not yet enabled as not yet fully tested
#$InterruptExpire = 0;

# number of nfprofile processes to spawn during the profiling phase
# depends on how busy your system is and how many CPUs you have
# on very busy systems increase it to a higher value
$PROFILERS = 2;

# if the PROFILEDATADIR is filled up to this percentage, a warning message will be printed.
# set to 0 to disable the test
$DISKLIMIT = 98;

# number of nfprofile processes to spawn during the profiling phase
$PROFILERS = 6;

# Netflow sources
# Define an ident string, port and colour per netflow source
#
# Required parameters:
#    ident   identifies this netflow source. e.g. the router name,
#            Upstream provider name etc.
#    port    nfcapd listens on this port for netflow data for this source
#            set port to '0' if you do not want a collector to be started
#    col     colour in nfsen graphs for this source
#
# Optional parameters
#    type    Collector type needed for this source. Can be 'netflow' or 'sflow'. Default is netflow
#    optarg  Optional args to the collector at startup
#
# Syntax:
#         'ident' => { 'port' => '<portnum>', 'col' => '<colour>', 'type' => '<type>' }
# Ident strings must be 1 to 19 characters long only, containing characters [a-zA-Z0-9_].

%sources = (
{% for source in nfsen_sources %}
{{ source }}
{% endfor %}
{% if not nfsen_sources %}
# Examples:
#    'upstream1'    => { 'port' => '9995', 'col' => '#0000ff', 'type' => 'netflow' },
#    'peer1'        => { 'port' => '9996', 'IP' => '172.16.17.18' },
#    'peer2'        => { 'port' => '9996', 'IP' => '172.16.17.19' },
{% endif %}
);

#
# Low water mark: When expiring files, delete files until
# size = $low_water % of max_size
# typically 90
$low_water = 90;

#
# syslog facility for periodic jobs
# nfsen uses level 'debug', 'info', 'warning' and 'err'
# Note: nfsen is very chatty for level 'debug' and 'info'
# For normal operation, you may set the logging level in syslog.conf
# to warning or error unless you want to debug NfSen
$syslog_facility = 'local3';

#
# SYSLOG mess
# Log socket type: Most *NIX such as LINUX and *BSD are fine with 'unix'
# which is the default. You need to change that to 'stream' or 'inet' for
# some Solaris version 8/9, AIX and others ..
# You may set it to undef to prevent calling Sys::Syslog::setlogsock at all
# ( works for Solaris 10 and newer Sys::Syslog module
#
# If not defined at all, 'unix' is assumed unless for Solaris, which defaults to 'stream'
# $LogSocket = 'unix';

#
# Plugins
# Plugins extend NfSen for the purpose of:
# Periodic data processing, alerting-condition and alerting-action
# For data processing a plugin may run for any profile or for a specific profile only.
#     Syntax: [ 'profile list', 'module' ]
#            profile list:  ',' separated list of profiles ( 'profilegroup/profilename' ),
#                           or '*' for any profile, '!' for no profile
#            module:        Perl Module name, equal to plugin name
# The profile list '!' make sense for plugins, which only provide alerting functions
#
# The module follows the standard Perl module conventions, with at least one
# function: Init(). See demoplugin.pm for a simple template.
#
# A file with the same name in the FRONTEND_PLUGINDIR and .php extension is automatically
# recongized as frontend plugin.
#
# Plugins are installed under
# $BACKEND_PLUGINDIR and $FRONTEND_PLUGINDIR

@plugins = (
    # profile    # module
    # [ '*',     'demoplugin' ],
);

%PluginConf = (
    # For plugin demoplugin
    demoplugin => {
        # scalar
        param2 => 42,
        # hash
        param1 => { 'key' => 'value' },
    },
    # for plugin otherplugin
    otherplugin => [
        # array
        'mary had a little lamb'
    ],
);

#
# Alert module: email alerting:
# Use this from address
$MAIL_FROM   = 'your@from.example.net';

# Use this SMTP server
$SMTP_SERVER = 'localhost';

# Use this email body:
# You may have multiple lines of text.
# Var substitution:
# @alert@       replaced by alert name
# @timeslot@    replaced by timeslot alert triggered
$MAIL_BODY = q{
Alert '@alert@' triggered at timeslot @timeslot@
};

######################################################
#
# For the NfSen simulator include the section below.
#
######################################################
#
# Nfsen Simulator
# The simulator requires, that you have already installed
# and configured NfSen. The simulation is based on already
# pre-colleted data, which you may get from another live
# NfSen system.
#
# Steps to setup the NfSen simulator:
# 1. Configure the sources of the live profile with the
#    same names of the NfSen system, you take netflow data
#    for the simulation. Set the port for each netflow source
#    to 0 to prevent a collector to be started.
#    Install NfSen with this config in a seperate directory
# 2. Copy the pre-collected data into the appropriate
#    netflow directory of the live profile.
# 3. Configure the simulator using the parameters below
#    Enable Simulation mode => $SIMmode = 1
#    Configure the time window of the pre-collected data.
#      tstart    => Start of time window. yyyymmddhhmm
#      tbegin    => Optional parameter. Start of simulation
#                   profile exists already between tstart - tbegin
#      tend      => End of time window. yyyymmddhhmm
#      cycletime => simulation time in seconds of a 5min slot
#    Setting cycletime = 0 processes the cycles as fast as
#    possible. Please note, if you test plugings, your
#    cycletime needs to be at least the time required to
#    process all plugins.
# 4. Start nfsen: ../nfsen start
#    Simulation starts
#
# The simulator runs from tstart to tend and stops when tend
# is reached. You may stop the simulation at any given time
# using ./nfsen stop. To continue the simulation start NfSen
# again: ./nfsen start. You may reset the simulator at any
# given time using ./nfsen abort-reset. This stops the sumulation
# and rolls back to tstart. All profiles/alerts are deleted,
# so you may start from scratch again.
#
# Configure simulator parameters
#
# $SIMmode = 1;
# %sim = (
#    'tstart'     => '200707100000',    # Simulation data available from July 10th 2007 00:00
#    'tbegin'     => '200707110000',    # Simulation begins at July 11th 2007 00:00
#    'tend'       => '200707112355',    # Simulation ends at July 11th 2007 23:55
#    'cycletime'  => '30',              # 30s per 5min slot
# );

1;

now create nfsen systemd startup script named nfsen.service as:

(venv) vmX-gY@ansible-host:~/ansible-playbook$ mkdir -p files/nfsen
(venv) vmX-gY@ansible-host:~/ansible-playbook$ vi files/nfsen/nfsen.service

[Unit]
Description=NfSen Service
After=network.target

[Service]
Type=forking
PIDFile=/var/nfsen/var/run/nfsend.pid
ExecStart=/var/nfsen/bin/nfsen start
ExecStop=/var/nfsen/bin/nfsen stop
Restart=on-abort

[Install]
WantedBy=multi-user.target

Now run ansible playbook to install nfsen as following.

(venv) vmX-gY@ansible-host:~/ansible-playbook$ ansible-playbook nfdump-nfsen.yml -t build-nfsen

PLAY [nfsen_hosts] ******************************************************************************************************************************************

TASK [Gathering Facts] **************************************************************************************************************************************
ok: [vmX-gY.lab.workalaya.net]

TASK [ensure package cache is up to date] *******************************************************************************************************************
[WARNING]: Could not find aptitude. Using apt-get instead

ok: [vmX-gY.lab.workalaya.net]

TASK [check for nfsen install directory] ********************************************************************************************************************
ok: [vmX-gY.lab.workalaya.net]

TASK [download nfsen source] ********************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [cleanup nfsen source directory] ***********************************************************************************************************************
ok: [vmX-gY.lab.workalaya.net]

TASK [extract nfsen source] *********************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [download nfsen patch] ********************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [apply patches to nfsen source] ************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net] => (item=nfsen.patch)

TASK [create nfsen.conf] ************************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [create nfsen user] ************************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [run nfsen installer] **********************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [install nfsen.service systemd] ************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [enable nfsen] *****************************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [start nfsen] ******************************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

PLAY RECAP **************************************************************************************************************************************************
vmX-gY.lab.workalaya.net   : ok=14   changed=10   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

View flows via the web:

You can find the nfsen page here:

http://vmX-gY.lab.workalaya.net/nfsen/nfsen.php

or, if you are using a single nfSen instance:

http://srv1-gY.lab.workalaya.net/nfsen/nfsen.php

If you are working in pairs, then both of you should point your web browser to the PC which is receiving flows.

You may see a message such as:

Frontend - Backend version mismatch!

This will go away if you reload the page, it's not a problem.

Done! Move on to the third lab, exercise3-nfsen-top-talkers

NOTES

Adding sources

If you had multiple routers in your network all sending flows to the same collector, you can either configure them to send to different ports on the collector, or you can tell nfsen the source IP address of each router. This allows nfsen to show distinct data from each source.

DON'T DO THIS NOW as you only have a single router, but if you needed to, you would do it as follows: