Skip to content

Developing NetDoc

Jump to bottom
Andrea Dainese edited this page Mar 27, 2024 · 6 revisions

You are invited to contribute to the improvement of NetDoc. Before starting, please open a issue in the NetDoc repository and discuss your idea.

To begin, you need to fork the NetDoc repository on GitHub.

Next, set up your local environment as follows:

mkdir ~/src
cd ~/src
ln -s /opt/netbox
git clone https://github.com/USERNAME/netdoc
git checkout -b NEWBRANCH

After executing the above commands, ensure that:

  • ~/src/netbox points to your local NetBox installation.
  • ~/src/netdoc points to your forked NetDoc repository.

Replace the following placeholders in the script:

  • USERNAME: replace with your GitHub username.
  • NEWBRANCH: replace with the name of the branch you are creating.

Please note that a local NetBox configuration with the NetDoc plugin installed is assumed. Refer to the third chapter on how to install NetBox with NetDoc.

Install NetDoc from a local directory

Within the third chapter, NetDoc is initially installed from the official PyPI repository. However, for development purposes, it is necessary to utilize a locally available version of NetDoc.

sudo -u netbox echo "-e ${HOME}/src/netdoc" > /opt/netbox/local_requirements.txt
sudo -u netbox /opt/netbox/upgrade.sh

To circumvent potential permission conflicts, it is advisable to reinstall NetBox using your individual user account when performing a development installation.

Starting NetBox

During the development process, it is essential to monitor logs in real-time and frequently restart daemons. As a common practice, I am accustomed to manually initiating daemons from the command-line interface (CLI).

/opt/netbox/venv/bin/python3 manage.py runserver 0.0.0.0:8000 --insecure
/opt/netbox/venv/bin/python3 manage.py rqworker high default low

Testing

Before committing a pull request, it is crucial to test your environment. Testing locally will save you time.

The PostgreSQL user configured in NetBox must have the privilege to create new databases:

sudo -u postgres psql
alter user netbox createdb;

Install additional prerequisites:

sudo apt install pre-commit
/opt/netbox/venv/bin/pip install pylint

Lint your code:

cd ~/src/netdoc
pre-commit run --all-files
/opt/netbox/venv/bin/pylint --rcfile=.pylintrc netdoc

Test your code:

/opt/netbox/venv/bin/python3 /opt/netbox/netbox/manage.py test netdoc --keepdb

If everything is functioning correctly, you can proceed to commit your code and create a pull request.

Adding Support for New Devices

In order to support new devices, the following steps should be taken:

For further guidance and as an example, please refer to this pull request.

Adding stencils for diagrams

In order to support new stencils, the following steps should be taken:

For further guidance and as an example, please refer to this pull request.

Adding diagrams

In order to support new diagrams, the following steps should be taken:

  • Define new diagram types into DiagramModeChoices on models.py.
  • Add the functions to get the diagram data and to export it in Draw.io format into netdoc/utls.py: in the example linked below get_site_topology_data and get_site_drawio_topology implement the new site diagram type.

For further guidance and as an example, please refer to this pull request.

Debugging

Using and developing NetDoc requires to debug disvoery, parser, and ingestor scripts. In order, follow the below paragraphs.

Discovery script

Manually discover one or more devices:

from netdoc import tasks

tasks.discovery(["172.25.82.34","172.25.82.39","172.25.82.40"])

NTC Templates

Manually parse downloaded logs using official NTC templates:

import textfsm
import pprint

template_file = 'ntc_templates/templates/cisco_xr_show_ipv4_interface.textfsm'
raw_output_file = 'tests/cisco_xr/show_ipv4_interface/cisco_xr_show_ipv4_interface.raw'

with open(template_file) as fd_t, open(raw_output_file) as fd_o:
    re_table = textfsm.TextFSM(fd_t)
    parsed_header = re_table.header
    parsed_output = re_table.ParseText(fd_o.read())

pprint.pprint(parsed_header)
pprint.pprint(parsed_output)

If you find errors on NTC tempaltes, please open an issue on NTC.

Parser scripts

Manually parse NetDoc logs using official NTC templates:

from netdoc import models
import importlib
from netdoc import functions
import logging
import pprint

request = "show vrf"
mode = "netmiko_cisco_nxos"
request = "show ip interface"
mode = None

logs = models.DiscoveryLog.objects.all()

request = None # or "show vrf"
mode = None    # or "netmiko_cisco_nxos"

if mode:
    logs = logs.filter(discoverable__mode=mode)
if request:
    logs = logs.filter(request=request)
logs = logs.filter(success=True)

for log in logs:
    try:
        functions.log_parse(log)
    except:
        pass
	print('Command: ', log.command)
	print('ID: ', log.id)
    print('Address: ', log.discoverable.address)
	print('Device: ', log.discoverable.device)
    print('Parsed: ', log.parsed)
    print('Items: ', len(log.parsed_output))
    pprint.pprint(log.parsed_output)
    print('-' * 70)

Ingestor scripts

Manually ingest NetDoc parsed logs using official NTC templates:

from netdoc import models
import importlib
from netdoc.ingestors import functions
import logging

request = None # or "show vrf"
mode = None    # or "netmiko_cisco_nxos"

logs = models.DiscoveryLog.objects.all()
if mode:
    logs = logs.filter(discoverable__mode=mode)
if request:
    logs = logs.filter(request=request)
logs = logs.filter(parsed=True)

for log in logs:
        try:
            functions.log_ingest(log)
        except functions.NoIngestor:
            pass
        except functions.Postponed as err:
            print(err)

Introduction | Device configuration | NetBox installation | Utilizing NetDoc | Developing NetDoc | FAQ | Issues | Sponsorship

Clone this wiki locally