Utilizing NetDoc

This chapter provides a comprehensive guide on how to utilize NetDoc step by step.

Credentials

To access discoverable devices, it is necessary to add the appropriate credentials. Go to NetDoc -> Credentials and add a new credential:

• Name: the name of the credential.
• Username: The username (if required) used to log in to the device.
• Password: The password (if required) used to log in to the device.
• Enable password: The password (if required) to elevate privileges.

Credentials are encrypted using SECRET_KEY defined in the configuration file.

First discovery

Once you have configured at least one credential, you can initiate the discovery process. Go to Other -> Scripts -> NetDoc -> Add and discover and configure your discovery parameters:

• Credential: select the credential to be used for accessing discoverable devices.
• Mode: select the mode (type) of the discoverables. The discovery process is controlled by the respective modes.
• Site: select the site to be associated with the discoverables.
• IP addresses: add the IP addresses to be discovered.
• Filters: set the commands to include or exclude (e.g. "bgp,arp,mac,route").
• Filter type: set to exclude commands matching the filters (exclude) or to include commands matching the filter only (include).
• Schedule at: set the date if you want to schedule the discovery or leave it empty to run immediately.
• Recurs every: leave it empty.
• Commit changes: set it to False for a dry run.

Go to Other -> Jobs and monitor the job status:

• NetDoc.AddDiscoverable: indicates the discovery phase, including device login and executed commands. Once this job is completed, an ingest job is initiated.
• NetDoc.Ingest: represents the ingestion phase. In this phase, the outputs from discoverable devices are parsed, formatted, and added to the NetBox database. Ingest jobs may spawn additional jobs of the same type.

The behavior of the discovery and ingestion phases is determined by the parameters configured in the configuration.py file:

• MAX_INGESTED_LOGS: specifies the number of ingested logs before initiating a new ingestion job (default is 50).
• NTC_TEMPLATES_DIR: defines the path where NTC templates are installed (e.g., /opt/ntc-templates/ntc_templates/templates).
• NORNIR_LOG: specifies the location where the nornir.log file is saved (default is the NetBox directory, e.g., /opt/netbox/netbox/nornir.log).
• NORNIR_TIMEOUT: specifies the maximum time in seconds to wait for each discovery (default is 300).
• RAISE_ON_CDP_FAIL: determines whether an exception should be raised when managing CDP output (default is True).
• RAISE_ON_LLDP_FAIL: determines whether an exception should be raised when managing LLDP output (default is True).

CDP and LLDP details can vary. In some cases, NetDoc may detect multiple neighbors attached to a single wire. By default, NetDoc raises an exception in such cases. To bypass the exception, set both RAISE_ON_CDP_FAIL and RAISE_ON_LLDP_FAIL to False.

After the jobs are completed, check the discoverable list:

• The manually added discoverables should have devices attached and display some discovery logs.
• The devices discovered via CDP and LLDP will be shown with an IP address but require additional information to be fully discovered (see the Discoverable section).

At this point, the NetBox database should be populated. If not, please refer to the Discovery logs section.

Discovery logs

Once the ingestion phase is complete, all relevant data should be successfully ingested into NetBox. Go to NetDoc -> Logs and monitor the log status:

• Configuration: stores the running configuration. These logs are not supported or ingested.
• Supported: only supported logs are parsed and ingested. Unsupported logs can still provide details from discoverable devices, even if they are not ingested into NetBox.
• Success: command execution is considered failed when the output triggers a RegEx defined in FAILURE_OUTPUT. Only success logs are parsed.
• Parsed: only success logs are parsed using different frameworks like NTC Templates. Parsing is controlled by the respective parse method.
• Ingested: parsed logs are ingested into NetBox. Ingestion is controlled by the ingestion scripts.

Discoverable

Discoverables can be manually added or edited. Editing is required to discover devices found via CDP or LLDP. Go to NetDoc -> Discoverables and add or edit a discoverable:

• Address: the IP address or FQDN used to reach the device.
• Credential: the credential used to login to the device.
• Mode: select the mode (type) of the discoverable. The discovery process is controlled by the respective mode.
• Discoverable: set it to True if the device should be discovered.
• Site: select the site to be associated with the device.

Manual discovery

Devices can be discovered using various methods.

Go to NetDoc -> Discoverables and select one or more discoverables and click the Discover Selected button.

Go to NetDoc -> Discoverables, open a discoverable and click the Discover button.

Go to Other -> Scripts -> NetDoc -> Discovery:

• Discoverables: choose one or more devices in the discoverables field. If left empty, all devices will be selected for discovery..
• Undiscovered only: check the box if you want to select only undiscovered devices for discovery..

Devices with discoverable = False will be skipped during the discovery process.

Manual ingestion

Logs can be manually ingested or re-ingested. Go to Other -> Scripts -> NetDoc -> Ingest and start the script:

• Discoverables: choose one or more devices. If left empty, all devices will be selected for ingestion.
• Re ingest: check this box if you want to re-ingest all previously ingested logs.
• Max ingested logs: divide the job into multiple tasks.

Post discovery

After completing the discovery and ingestion phases, it is necessary to perform the following review tasks using a technical formal language:

• Logs: in NetDoc -> Logs, check for errors in the supported logs (success = False, parsed = False, ingested = False).
• Devices: in Devices -> Devices, review the ingested devices. Pay close attention to the interfaces and IP addresses associated with each device.
• Cables: in Connections -> Cables, review the ingested cables.
• IP Addresses: in IPAM -> IP ADDRESSES -> IP Addresses and review the ingested IP addresses.
• Prefixes: in IPAM -> PREFIXES -> Prefixes and review the prefixes. Ensure that the IP network is correctly associated with each IP address.
• VRFs: in IPAM -> VRFS-> VRFs and review the ingested VRFs. Check for missing or duplicated VRFs.
• VLANs: in IPAM -> VLANS -> VLANs and review the ingested VLANs. Look for any missing or duplicated VLANs.

It's important to note that if a trunk is configured to pass VLANs such as 1,1000-3000, it will ingest 2001 VLANs even if they are not being used. This behavior is due to NetBox allowing interfaces to be configured to trunk all VLANs or specify them individually.

Fixing IP Addresses and VRFs

VRFs are attributes local to each device but can be perceived globally. This can result in ingestion errors in the following scenarios:

• The local interface may be associated with the default VRF, but the network is associated and exported using a specific VRF.
• The local interface may be configured with the VRF name as VRFGREEN, even though the correct name is GREEN.

After completing the discovery and ingestion phases, a network administrator should review the IP addresses and prefixes and fix any VRF-related issues.

Got to IPAM -> PREFIXES -> Prefixes, delete any duplicated entries, and fix the VRF assignments:

The screenshot above illustrates four duplicated VRFs that should be deleted.

Go to IPAM -> IP ADDRESSES -> IP Addresses, delete any duplicated entries, and correct the VRF assignments:

The screenshot above displays two IP addresses that are assigned to the wrong VRF.

Please note that any changes made will no longer be overwritten by NetDoc.

Finally go to Other -> Reports, run VRFIpPrefixIntegrityCheck and verify if any errors are reported:

The screenshot above highlights certain IP addresses belonging to the global VRF that do not correspond to any prefix. This discrepancy occurs because the IP addresses are associated with a specific VRF rather than the global VRF.

Network diagrams

NetDoc enables the creation of network diagrams. Go to NetDoc -> DIAGRAMS -> Diagrams and add a new network diagram:

• Name: specify a name for the network diagram.
• Mode: select either L2 for a layer 2 (cables) diagram or L3 for a network (routing) diagram.
• Device roles: limit the network diagram to specific device roles. Leave this field empty to include all device roles.
• Sites: limit the network diagram to specific sites. Leave this field empty to include all sites.
• Vrfs: limit the network diagram to specific VRFs. Leave this field empty to include all VRFs.
• Include global vrf: set this field and leave the previous field empty to include only the global VRF.

Once the new diagram is added, go to NetDoc -> DIAGRAMS -> Diagrams and open the new diagram:

Please keep in mind the following details:

• The node icons in the diagram depend on the device roles. By default, devices are created using a generic Unknown role.
• Node positions in the diagram are random and dynamic. Use the Static button to select and fix positions, then press Save.
• To download the diagram in Draw.io format, click on Export.

The stencils used in NetDoc diagrams are controlled by DeviceImageChoices (NetDoc diagram) and DRAWIO_ROLE_MAP. These stencils are courtesy of the Draw.io.

To automatically create all required device roles, access the Scripts section under Other and run the script named Create device roles. Then, go to Devices -> DEVICES -> Devices and update the device role.

Maintenance jobs

There are additional considerations regarding the usage of NetDoc for maintenance purposes:

• Discovery can be scheduled to run continuously. Go to Other -> Scripts -> Discover and configure a recurrent job for the Discover script.
• The IPAM can be updated from ARP tables. Before proceeding, ensure that you have checked the VRFIpPrefixIntegrityCheck report (mentioned earlier). Go to Other -> Scripts -> Update IPAM from ARP tables and configure a recurrent job.
• It is recommended to delete old discovery logs, ARP entries, and MAC address entries periodically. Go to Other -> Scripts -> Discover and configure a recurrent job.