Netdot

Introduction

The Network Documentation Tool (Netdot) is an open source software designed to help network administrators collect, organize and maintain network documentation. Netdot was developed at the University of Oregon.

Goals

• In these exercises we will install Netdot and demonstrate some of its most important features.

Notes

• Commands preceded with "$" imply that you should execute the command as a general user - not as root.
• Commands preceded with "#" imply that you should be working as root.
• Commands with more specific command lines (e.g. "rtr1-gY>" or "mysql>") imply that you are executing commands on remote equipment, or within another program.

Installation

Netdot may already be installed in your campus server. Ask the instructor.

Now login back to ansible control host.

Create a new ansible playbook named netdot.yml to install netdot. In this playbook we will perform following tasks:

• install dependencies required for netdot build from source
• clone netdot repository (source)
• patch netdot installation source
• build netdot dependencices
• initialize the netdot site configuration
• install the application
• initialize the netdot database
• configure apache2
• install netdot cron job

(venv) vmX-gY@ansible-host:~/ansible-playbook$ vi netdot.yml

- hosts: netdot_hosts
  become: true
  tasks:
    - name: ensure package cache is up to date
      apt: update_cache=yes cache_valid_time=3600
      tags: install-base

    - name: install netdot required packages
      package:
        name: "{{ item }}"
        state: present
      with_items:
        - build-essential
        - mysql-server
        - mysql-client
      tags: install-base

    - name: check for netdot install directory
      stat:
        path: /usr/local/netdot
      register: netdot_check
      tags: install

    - name: netdot
      block:
        - name: check local netdot source exist
          git:
            # repo: https://github.com/cvicente/Netdot.git
            repo: http://www.lab.workalaya.net/git/Netdot.git
            dest: /usr/local/src/Netdot
            clone: no
            update: no
          register: netdot_source

        - name: fetch netdot source
          git:
            # repo: https://github.com/cvicente/Netdot.git
            repo: http://www.lab.workalaya.net/git/Netdot.git
            dest: /usr/local/src/Netdot
          when: netdot_source.changed

        # next three tasks plus the netdot-perldeps.patch patch are a hack.
        # they disable the call to netdisco-mibs-downloader and pull the content
        # manually to where netdisco-mibs-installer expects them.
        # Two reasons:
        #     - sourceforge.net was down and required javascript
        #     - we wanted to reduce time and decrease bandwidth usage all hosts from www.lab.workalaya.net for downloads

        - name: delete netdisco tmp dir
          file: path=/tmp/netdisco-mibs state=absent

        - name: create netdisco tmp dir
          file: path=/tmp/netdisco-mibs state=directory owner=root

        - name: fetch netdisco mibs
          get_url:
            # url: 'https://sourceforge.net/projects/netdisco/files/netdisco-mibs/1.0/netdisco-mibs-1.0.tar.gz/download'
            url: 'http://www.lab.workalaya.net/downloads/netdot/netdisco-mibs-1.0.tar.gz'
            dest: '/tmp/netdisco-mibs/netdisco-mibs-1.0.tar.gz'
          retries: 5
          delay: 10

        - name: copy patches
          copy:
            src: 'files/netdot/{{ item }}'
            dest: '/usr/local/src/Netdot/{{ item }}'
          with_items:
            - netdot-perldeps.patch

        - name: apply patches
          shell: patch -p0 < {{item}}
          args:
            chdir: '/usr/local/src/Netdot'
          with_items:
            - netdot-perldeps.patch
          ignore_errors: true

        - name: patch netdot for ubuntu 18.0x version
          replace:
            path: "/usr/local/src/Netdot/bin/perldeps.pl"
            replace: "{{ item.replace }}"
            regexp: "{{ item.search }}"
          with_items:
            - { search: "dnssec-tools", replace: "libnet-dns-zonefile-fast-perl" }
            - { search: "deb http", replace: "deb [allow-insecure=yes] http" }
            - { search: "deb-src http", replace: "deb-src [allow-insecure=yes] http" }

        - name: build netdot pre-requisites
          shell: echo -e "mysql\ny\ny\ny\n" | make apt-install
          args:
            chdir: '/usr/local/src/Netdot'

        - name: apply patches to Perl DBI.pm
          shell: patch /usr/share/perl5/Class/DBI.pm  < DBI.pm.patch
          args:
            chdir: '/usr/local/src/Netdot/patches/ubuntu-1604/'
          ignore_errors: true

        - name: copy Default.conf to Site.conf
          command: cp etc/Default.conf etc/Site.conf
          args:
            chdir: '/usr/local/src/Netdot/'
            creates: 'etc/Site.conf'

        - name: edit Site.conf before building
          lineinfile:
          args:
            dest: '/usr/local/src/Netdot/etc/Site.conf'
            regexp: '{{ item.regexp }}'
            line: '{{ item.line }}'
            state: present
          with_items:
            - { regexp: "^NETDOTNAME", line: "NETDOTNAME => '{{inventory_hostname}}'," }
            - { regexp: "^DB_DBA_PASSWORD", line: "DB_DBA_PASSWORD =>  '{{class_password}}'," }
            - { regexp: "^DEFAULT_SNMPCOMMUNITIES", line: "DEFAULT_SNMPCOMMUNITIES  =>  ['NetManage', 'public']," }
            - { regexp: "^NMS_DEVICE", line: "NMS_DEVICE => '{{inventory_hostname}}'," }
            - { regexp: "^DEFAULT_DNSDOMAIN", line: "DEFAULT_DNSDOMAIN  => 'lab.workalaya.net'," }
            - { regexp: "^DEVICE_NAMING_METHOD_ORDER", line: "DEVICE_NAMING_METHOD_ORDER => [ 'sysname', 'snmp_target' ]," }

        - name: insert database schema
          command: make installdb
          args:
            chdir: /usr/local/src/Netdot/
          ignore_errors: yes

        - name: install netdot
          command: make install APACHEUSER=www-data APACHEGROUP=www-data
          args:
            chdir: /usr/local/src/Netdot/
            creates: /usr/local/netdot

        - name: add netdot configuration to apache
          file:
            src: /usr/local/netdot/etc/netdot_apache24_local.conf
            dest: /etc/apache2/conf-available/netdot.conf
            state: link

        - name: enable netdot apache24 config
          command: a2enconf netdot
          args:
            creates: /etc/apache2/conf-enabled/netdot.conf
          notify: restart apache2

        - name: install cron job
          command: cp netdot.cron /etc/cron.d/netdot
          args:
            chdir: '/usr/local/src/Netdot/'
            creates: /etc/cron.d/netdot
      when: netdot_check.stat.isdir is not defined or netdot_check.stat.isdir == false
      tags: install

  handlers:
    - name: restart apache2
      service:
        name: apache2
        state: restarted

update inventory/hosts as following

(venv) vmX-gY@ansible-host:~/ansible-playbook$ vi inventory/hosts

[nagios_hosts]
vmX-gY.lab.workalaya.net

[snmp_hosts]
vmX-gY.lab.workalaya.net

[smokeping_hosts]
vmX-gY.lab.workalaya.net

[netdot_hosts]
vmX-gY.lab.workalaya.net

create patch files named netdot-perldeps.patch as

(venv) vmX-gY@ansible-host:~/ansible-playbook$ mkdir -p files/netdot
(venv) vmX-gY@ansible-host:~/ansible-playbook$ vi files/netdot/netdot-perldeps.patch

copy and paste following text and save it.

-- bin/perldeps.pl.orig        2016-07-13 10:16:03.717682539 +0000
+++ bin/perldeps.pl     2016-07-13 10:08:24.358430631 +0000
@@ -232,8 +232,8 @@
        my $ans = <STDIN>;
        if ( $ans =~ /(Y|y)/ ){
            print "Downloading necessary SNMP MIB files. This may take a few minutes.\n";
-           &cmd('rm -fr /tmp/netdisco-mibs');
-           &cmd('/usr/sbin/netdisco-mibs-download');
+           #&cmd('rm -fr /tmp/netdisco-mibs');
+           #&cmd('/usr/sbin/netdisco-mibs-download');
            print "\nInstalling SNMP MIB files\n";
            &cmd('/usr/sbin/netdisco-mibs-install');
            print "\nA new /etc/snmp/snmp.conf needs to be installed to point to the newly ".

or click here to download netdot-perldeps.patch

Now run ansible playbook to install netdot as following.

NOTE: it takes long time to complete this task

(venv) vmX-gY@ansible-host:~/ansible-playbook$ ansible-playbook netdot.yml

PLAY [netdot_hosts] ******************************************************************************************************************************************

TASK [Gathering Facts] ***************************************************************************************************************************************
ok: [vmX-gY.lab.workalaya.net]

TASK [ensure package cache is up to date] ********************************************************************************************************************
[WARNING]: Could not find aptitude. Using apt-get instead

ok: [vmX-gY.lab.workalaya.net]

TASK [install netdot required packages] **********************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net] => (item=build-essential)
changed: [vmX-gY.lab.workalaya.net] => (item=mysql-server)
changed: [vmX-gY.lab.workalaya.net] => (item=mysql-client)

TASK [check for netdot install directory] ********************************************************************************************************************
ok: [vmX-gY.lab.workalaya.net]

TASK [check local netdot source exist] ***********************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [fetch netdot source] ***********************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [delete netdisco tmp dir] *******************************************************************************************************************************
ok: [vmX-gY.lab.workalaya.net]

TASK [create netdisco tmp dir] *******************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [fetch netdisco mibs] ***********************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [copy patches] ******************************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net] => (item=netdot-perldeps.patch)

TASK [apply patches] *****************************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net] => (item=netdot-perldeps.patch)

TASK [patch netdot for ubuntu 18.0x version] *****************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net] => (item={'search': 'dnssec-tools', 'replace': 'libnet-dns-zonefile-fast-perl'})
changed: [vmX-gY.lab.workalaya.net] => (item={'search': 'deb http', 'replace': 'deb [allow-insecure=yes] http'})
changed: [vmX-gY.lab.workalaya.net] => (item={'search': 'deb-src http', 'replace': 'deb [allow-insecure=yes] http'})

TASK [build netdot pre-requisites] ***************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [apply patches to Perl DBI.pm] *****************************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [copy Default.conf to Site.conf] ************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [edit Site.conf before building] ************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net] => (item={'regexp': '^NETDOTNAME', 'line': "NETDOTNAME => 'vmX-gY.lab.workalaya.net',"})
changed: [vmX-gY.lab.workalaya.net] => (item={'regexp': '^DB_DBA_PASSWORD', 'line': "DB_DBA_PASSWORD =>  'lab',"})
changed: [vmX-gY.lab.workalaya.net] => (item={'regexp': '^DEFAULT_SNMPCOMMUNITIES', 'line': "DEFAULT_SNMPCOMMUNITIES  =>  ['NetManage', 'public'],"})
changed: [vmX-gY.lab.workalaya.net] => (item={'regexp': '^NMS_DEVICE', 'line': "NMS_DEVICE => 'vmX-gY.lab.workalaya.net',"})
changed: [vmX-gY.lab.workalaya.net] => (item={'regexp': '^DEFAULT_DNSDOMAIN', 'line': "DEFAULT_DNSDOMAIN  => 'lab.workalaya.net',"})
changed: [vmX-gY.lab.workalaya.net] => (item={'regexp': '^DEVICE_NAMING_METHOD_ORDER', 'line': "DEVICE_NAMING_METHOD_ORDER => [ 'sysname', 'snmp_target' ],"})

TASK [insert database schema] ********************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [install netdot] ****************************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [add netdot configuration to apache] ********************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [enable netdot apache24 config] *************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [install cron job] **************************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

RUNNING HANDLER [restart apache2] ****************************************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

PLAY RECAP ***************************************************************************************************************************************************
vmX-gY.lab.workalaya.net   : ok=22   changed=18   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Operation

Log into the web interface

In your browser, go to:

http://vmX-gY.lab.workalaya.net/netdot

Log in with username: admin and password: admin

Changing default passwords

Netdot comes with three default user accounts. You should change the default passwords on those.

Go to the "Contacts" tab, then search for "Admin". You should see the details for the Admin user. Click on [edit], and find the Password field. Type the password you used to log in to your server, then click on the "Update" button.

Repeat the same steps for the other default users:

• operator
• guest

Discovering devices

If you have not done so yet, configure SNMP on your server and your router.

Ask the instructor to provide you with instructions for configuring SNMP on Cisco routers and Linux

Now back to ansible control host. Let's create a file with all the devices in the lab network that respond to SNMP:

(venv) vmX-gY@ansible-host:~/ansible-playbook$ mkdir -p files/netdot
(venv) vmX-gY@ansible-host:~/ansible-playbook$ vi files/netdot/discoverme.txt

Copy and paste the list below. You will need to change "Y" to your group number

rtr1-gY.lab.workalaya.net
vm1-gY.lab.workalaya.net
vm2-gY.lab.workalaya.net
vm3-gY.lab.workalaya.net
srv1-gY.lab.workalaya.net

Now, lets create another ansible playbook to perform netdot to discover those devices as:

(venv) vmX-gY@ansible-host:~/ansible-playbook$ vi netdot-device-discovery.yaml

- hosts: netdot_hosts
  become: true
  tasks:
    - name: copy device list for auto discovery
      copy:
        src: files/netdot/discoverme.txt
        dest: /tmp/
      tags: discovery

    - name: run device discovery
      command: bin/updatedevices.pl -E /tmp/discoverme.txt -IAF
      args:
        chdir: /usr/local/netdot
      tags: discovery

now run ansible playbook as:

(venv) vmX-gY@ansible-host:~/ansible-playbook$ ansible-playbook netdot-device-discovery.yaml

PLAY [netdot_hosts] ************************************************************************************************************

TASK [Gathering Facts] *********************************************************************************************************
ok: [vmX-gY.lab.workalaya.net]

TASK [copy device list for auto discovery] *************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

TASK [run device discovery] ****************************************************************************************************
changed: [vmX-gY.lab.workalaya.net]

PLAY RECAP *********************************************************************************************************************
vmX-gY.lab.workalaya.net   : ok=3    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

When that is done, go to the web interface and navigate to

Management -> Devices

Leave the search box empty, and click on the "Find" button.

You should see all the discovered devices in that list. Go to the link for your group's router (e.g. rtr1-gY.lab.workalaya.net)

Navigate to all the tabs: Basic, Interfaces, Modules, IP Info, etc. Netdot allows you to augment the information gathered from the device with details entered manually.

In the ARP section, you should see one entry with a timestamp. Click on that entry. You should see a table associating IP addresses with MAC addresses. This is the ARP table discovered from rtr1-gY. You should see your servers's IP address and MAC address.

Finding a computer in your network

• Obtain the MAC address from your laptop (or desktop). Copy it in the clipboard.
• In the Netdot web interface, go to Management -> Devices
• Paste your MAC address and hit ENTER

Netdot will show you which devices were seeing that MAC address the last time that it discovered the network.

Managing IP address space

Go to Management -> Address Space

You should see a list of IP blocks. Some of these come pre-installed in Netdot, and some (Subnets) are discovered from routers.

• Click on 100.68.Y.0/24 (where Y is your group number)
• Click on [edit]
• In the Description field, type "Group Y Servers"
• Click "Save"

Create a container to include all the group subnets

In the section called "Address Space Tasks" on top, click on the "[new]" button and enter the following:

• IP/Prefix: 100.68.0/16
• Owner: click on [new].
• In the new "Entity" window, enter:
• Name: npNOG5 Lab
• Insert button, then [close]
• Used by: (leave blank)
• Status: Container
• Description: npNOG5 lab student networks
• Save

You should now see the new Container page. It shows a graphical representation of the block. All the existing subnets are shown in red. The green space represents unused or available address space.

• On the top of the graph there is a section called "Zoom: set one row equal to" Select /24 from the drop-down menu. Each row now represents a /24 block
• Click on [Tree View] to see a tree graph view of the IP hierarchy

Polling devices

Periodically you will want to connect again to your routers and switches to fetch their interface list, ARP and forwarding tables etc. You can run the command which does this:

login to your VM

lab@vm1-g1:~$ sudo /usr/local/netdot/bin/updatedevices.pl -DIFAT

• -D: poll all devices already in the database
• -I: get device info (e.g. sysName)
• -F: get switch forwarding tables
• -A: get router ARP tables
• -T: re-calculate the topology

To avoid having to run this by hand, you can install a crontab which will do it automatically at set times of day. We installed the crontab file in /etc/cron.d previously. If you look in /etc/cron.d/netdot you will see that this command executes once each hour by default.

More information

• Official Netdot Website https://github.com/cvicente/Netdot/
• Netdot Manual http://htmlpreview.github.io/?https://github.com/cvicente/Netdot/blob/master/doc/manual/netdot-manual.html