NXOS working, lint/unit tests up

README.md

@@ -16,18 +16,20 @@ granularity, it rapidly discovers the vast majority of OSPF problems.
   * [FAQ](#faq)
 
 ## Supported platforms
-Today, Cisco IOS/IOS-XE and IOS-XR are supported. The valid `device_type`
+Today, Cisco IOS/IOS-XE, IOS-XR, and NX-OS are supported. Valid `device_type`
 options used for inventory groups are enumerated below. Each platform
 has a folder in the `devices/` directory, such as `devices/ios/`. The
 file named `main.yml` is the task list that is included from the main
 playbook which begins the device-specific tasks.
 
   * `ios`: Cisco classic IOS and Cisco IOS-XE devices.
   * `iosxr`: Cisco IOS-XR devices.
+  * `nxos`: Cisco NX-OS devices.
 
 Testing was conducted on the following platforms and versions:
   * Cisco CSR1000v, version 16.07.01a, running in AWS
   * Cisco XRv9000, version 6.3.1, running in AWS
+  * Cisco 3172T, version 6.0.2.U6.4a, hardware appliance
 
 Control machine information:
 ```
@@ -235,14 +237,14 @@ Neighbor ID     Pri   State           Dead Time   Address         Interface
 ```
 
 ## FAQ
-__Q__: A lot of the code between IOS and IOS-XR is the same. Why not combine it?
-__A__: The goal is to support more platforms in the future, such as Nexus OS,
+__Q__: Most code between IOS, IOS-XR, and NX-OS is the same. Why not combine it?
+__A__: The goal is to support more platforms in the future such as Cisco
 ASA-OS, and possibly non-Cisco devices. These devices will likely return
 different sets of information. This tool is designed to be __simple__,
-not particularly advanced.
+not particularly advanced through layered abstractions.
 
 __Q__: Why not use an API like RESTCONF or NETCONF instead of SSH + CLI?
-__A__: This tool is designed for risk-averse users or networks that are not
+__A__: This tool is designed for risk-averse users or managers that are not
 rapidly migrating to API-based management. It is not an infrastructure-as-code
 solution and does not manage device configurations. All of the commands used
 in the playbook can be issued at privilege level 1 to further reduce risk.
@@ -257,3 +259,9 @@ level parameters for verification. Furthermore, the detailed statistics
 checking will alert the user to many errors (authentication, MTU mismatch, etc)
 at a more general level. The user can check the logs to see the exact commands,
 which includes the non-parsed interface text.
+
+__Q__: For NX-OS why didn't you use the `| json` filter from the CLI?
+__A__: While this would have saved a lot of parsing code, I did not want to
+have an inconsistent overall strategy for one network device. Additionally,
+the filter does not render milliseconds properly (eg, SPF throttle timers)
+which reduced my confidence in its overall accuracy.

group_vars/ospf_routers.yml

@@ -1,11 +1,11 @@
 ---
 all_areas:
-  #area0:
-    #type: standard  # valid values: standard, stub, nssa
-    #routers: 3
-    #drs: 1
-    #has_frr: true
-    #max_lsa3: 333
+  # area0:
+  #   type: standard  # valid values: standard, stub, nssa
+  #   routers: 3
+  #   drs: 1
+  #   has_frr: true
+  #   max_lsa3: 333
   area0:
     type: standard
     routers: 2

hosts.yml

@@ -1,5 +1,4 @@
 ---
-# group_vars/ospf_routers.yml
 all:
   hosts:
     localhost:

plugins/filter/filter.py

@@ -40,6 +40,13 @@ def filters():
 
     @staticmethod
     def _read_match(match, key_filler_list=None):
+        '''
+        Helper function which consumes a match object and an optional
+        list of keys to populate with None values if match is invalid.
+        Many operations follow this basic workflow, which iterates over
+        the items captured in the match, attempts to make them integers
+        whenever possible, and returns the resulting dict.
+        '''
         return_dict = None
         if match:
             return_dict = match.groupdict()
@@ -54,12 +61,16 @@ def _read_match(match, key_filler_list=None):
 
     @staticmethod
     def _get_match_items(pattern, text, extra_flags=0):
+        '''
+        Helper function that can perform iterative block matching
+        given a pattern and input text. Additional regex flags (re.DOTALL, etc)
+        can be optionally specified. Any fields that can be parsed as
+        integers are converted and the list of dictionaries containing the
+        matches of each block is returned.
+        '''
         regex = re.compile(pattern, re.VERBOSE + extra_flags)
         items = [match.groupdict() for match in regex.finditer(text)]
         for item in items:
-            # If there is an 'intf' key, make it lowercase
-            #if 'intf' in item:
-             #   item['intf'] = item['intf'].lower()
             for key in item.keys():
                 item[key] = FilterModule._try_int(item[key])
 
@@ -68,7 +79,7 @@ def _get_match_items(pattern, text, extra_flags=0):
     @staticmethod
     def nxos_ospf_traffic(text):
         '''
-        Parses information from the Cisco IOS-XR "show ip ospf traffic" command
+        Parses information from the Cisco NXOS "show ip ospf traffic" command
         family. This is useful for verifying various characteristics of
         an OSPF process/area statistics for troubleshooting.
         '''
@@ -112,11 +123,10 @@ def nxos_ospf_traffic(text):
     @staticmethod
     def nxos_ospf_dbsum(text):
         '''
-        Parses information from the Cisco IOS
+        Parses information from the Cisco NXOS
         "show ip ospf database database-summary" command family.
         This is useful for verifying various characteristics of
         an OSPF database to count LSAs for simple verification.
-        Note that this parser is generic enough to cover Cisco IOS-XR also.
         '''
         return_dict = {}
         process_pattern = r"""
@@ -156,6 +166,7 @@ def nxos_ospf_dbsum(text):
 
         return_dict.update({'areas': areas})
         return return_dict
+
     @staticmethod
     def nxos_ospf_neighbor(text):
         '''
@@ -172,25 +183,7 @@ def nxos_ospf_neighbor(text):
             (?P<peer>\d+\.\d+\.\d+\.\d+)\s+
             (?P<intf>[0-9A-Za-z./-]+)
         """
-        regex = re.compile(pattern, re.VERBOSE)
-        ospf_neighbors = []
-        for s in text.split('\n'):
-            m = regex.search(s)
-            if m:
-                d = m.groupdict()
-                d['priority'] = FilterModule._try_int(d['priority'])
-                d['state'] = d['state'].lower()
-                d['role'] = d['role'].lower()
-                d['intf'] = d['intf'].lower()
-
-                up_times = d['uptime'].split(':')
-                times = [FilterModule._try_int(t) for t in up_times]
-                upsec = times[0] * 3600 + times[1] * 60 + times[2]
-                d.update({'upsec': upsec})
-
-                ospf_neighbors.append(d)
-
-        return ospf_neighbors
+        return FilterModule._ospf_neighbor(pattern, text, ['uptime'])
 
     @staticmethod
     def nxos_ospf_basic(text):
@@ -237,7 +230,8 @@ def nxos_ospf_basic(text):
         areas = [match.groupdict() for match in regex.finditer(text)]
         for area in areas:
             area['num_intfs'] = FilterModule._try_int(area['num_intfs'])
-            area['id'] = FilterModule._try_int(ipaddress.IPv4Address(area['id_dd']))
+            converted_dd = ipaddress.IPv4Address(area['id_dd'])
+            area['id'] = FilterModule._try_int(converted_dd)
             if not area['type']:
                 area['type'] = 'standard'
             else:
@@ -275,25 +269,7 @@ def ios_ospf_neighbor(text):
             (?P<peer>\d+\.\d+\.\d+\.\d+)\s+
             (?P<intf>[0-9A-Za-z./-]+)
         """
-        regex = re.compile(pattern, re.VERBOSE)
-        ospf_neighbors = []
-        for s in text.split('\n'):
-            m = regex.search(s)
-            if m:
-                d = m.groupdict()
-                d['priority'] = FilterModule._try_int(d['priority'])
-                d['state'] = d['state'].lower()
-                d['role'] = d['role'].lower()
-                d['intf'] = d['intf'].lower()
-
-                dead_times = d['deadtime'].split(':')
-                times = [FilterModule._try_int(t) for t in dead_times]
-                deadsec = times[0] * 3600 + times[1] * 60 + times[2]
-                d.update({'deadsec': deadsec})
-
-                ospf_neighbors.append(d)
-
-        return ospf_neighbors
+        return FilterModule._ospf_neighbor(pattern, text, ['deadtime'])
 
     @staticmethod
     def ios_ospf_basic(text):
@@ -456,18 +432,18 @@ def ios_ospf_frr(text):
         """
         regex = re.compile(pattern, re.VERBOSE)
         frr_area_dict = {}
-        for s in text.split('\n'):
-            m = regex.search(s)
-            if m:
-                d = m.groupdict()
-                area = 'area' + d['id']
-                d['id'] = FilterModule._try_int(d['id'])
-                d['rlfa'] = d['rlfa'].lower() == 'yes'
-                d['tilfa'] = d['tilfa'].lower() == 'yes'
-                d['pref_pri'] = d['pref_pri'].lower()
-                d['topology'] = d['topology'].lower()
-
-                frr_area_dict.update({area: d})
+        for line in text.split('\n'):
+            match = regex.search(line)
+            if match:
+                gdict = match.groupdict()
+                area = 'area' + gdict['id']
+                gdict['id'] = FilterModule._try_int(gdict['id'])
+                gdict['rlfa'] = gdict['rlfa'].lower() == 'yes'
+                gdict['tilfa'] = gdict['tilfa'].lower() == 'yes'
+                gdict['pref_pri'] = gdict['pref_pri'].lower()
+                gdict['topology'] = gdict['topology'].lower()
+
+                frr_area_dict.update({area: gdict})
 
         return frr_area_dict
 
@@ -489,17 +465,17 @@ def ios_bfd_neighbor(text):
         """
         regex = re.compile(pattern, re.VERBOSE)
         bfd_neighbors = []
-        for s in text.split('\n'):
-            m = regex.search(s)
-            if m:
-                d = m.groupdict()
-                d['ld'] = FilterModule._try_int(d['ld'])
-                d['rd'] = FilterModule._try_int(d['rd'])
-                d['rhrs'] = d['rhrs'].lower()
-                d['state'] = d['state'].lower()
-                d['intf'] = d['intf'].lower()
-
-                bfd_neighbors.append(d)
+        for line in text.split('\n'):
+            match = regex.search(line)
+            if match:
+                gdict = match.groupdict()
+                gdict['ld'] = FilterModule._try_int(gdict['ld'])
+                gdict['rd'] = FilterModule._try_int(gdict['rd'])
+                gdict['rhrs'] = gdict['rhrs'].lower()
+                gdict['state'] = gdict['state'].lower()
+                gdict['intf'] = gdict['intf'].lower()
+
+                bfd_neighbors.append(gdict)
 
         return bfd_neighbors
 
@@ -518,7 +494,7 @@ def check_bfd_up(bfd_nbr_list, ospf_nbr):
                 is_up = bfd_nbr['state'] == 'up' and bfd_nbr['rhrs'] == 'up'
                 return is_up
 
-        raise ValueError('Peer {0} not found in bfd_nbr_list'.format(ospf_nbr['peer']))
+        raise ValueError('{0} not in bfd_nbr_list'.format(ospf_nbr['peer']))
 
     @staticmethod
     def iosxr_ospf_neighbor(text):
@@ -537,28 +513,40 @@ def iosxr_ospf_neighbor(text):
             (?P<uptime>[0-9:]+)\s+
             (?P<intf>[0-9A-Za-z./-]+)
         """
+        return FilterModule._ospf_neighbor(
+            pattern, text, ['deadtime', 'uptime'])
+
+    @staticmethod
+    def _ospf_neighbor(pattern, text, time_keys=None):
+        '''
+        Helper function specific to OSPF neighbor parsing. Each device type
+        is slightly different in terms of the information provided, but
+        most fields are the same. The time_keys parameter is a list of keys
+        which are expected to have values in the format "hh:mm:ss". These
+        are commonly uptime, deadtime, etc ... and are most useful when
+        converted into seconds as an integer for comparative purposes.
+        '''
         regex = re.compile(pattern, re.VERBOSE)
         ospf_neighbors = []
-        for s in text.split('\n'):
-            m = regex.search(s)
-            if m:
-                d = m.groupdict()
-                d['priority'] = FilterModule._try_int(d['priority'])
-                d['state'] = d['state'].lower()
-                d['role'] = d['role'].lower()
-                d['intf'] = d['intf'].lower()
-
-                dead_times = d['deadtime'].split(':')
-                times = [FilterModule._try_int(t) for t in dead_times]
-                deadsec = times[0] * 3600 + times[1] * 60 + times[2]
-                d.update({'deadsec': deadsec})
-
-                up_times = d['uptime'].split(':')
-                times = [FilterModule._try_int(t) for t in up_times]
-                upsec = times[0] * 3600 + times[1] * 60 + times[2]
-                d.update({'upsec': upsec})
-
-                ospf_neighbors.append(d)
+        for line in text.split('\n'):
+            match = regex.search(line)
+            if match:
+                gdict = match.groupdict()
+                gdict['priority'] = FilterModule._try_int(gdict['priority'])
+                gdict['state'] = gdict['state'].lower()
+                gdict['role'] = gdict['role'].lower()
+                gdict['intf'] = gdict['intf'].lower()
+
+                # If time keys is specified, iterate over the keys and perform
+                # the math to convert hh:mm:ss to an integer of summed seconds.
+                if time_keys:
+                    for k in time_keys:
+                        times = gdict[k].split(':')
+                        parts = [FilterModule._try_int(t) for t in times]
+                        totalsec = parts[0] * 3600 + parts[1] * 60 + parts[2]
+                        gdict.update({k + '_sec': totalsec})
+
+                ospf_neighbors.append(gdict)
 
         return ospf_neighbors
 
@@ -572,11 +560,14 @@ def iosxr_ospf_basic(text):
         return_dict = {}
 
         process_pattern = r"""
-            Routing\s+Process\s+"ospf\s+(?P<id>\d+)"\s+with\s+ID\s+(?P<rid>\d+\.\d+\.\d+\.\d+)
+            Routing\s+Process\s+"ospf\s+(?P<id>\d+)"\s+with\s+ID\s+
+            (?P<rid>\d+\.\d+\.\d+\.\d+)
             .*
             \s*Initial\s+SPF\s+schedule\s+delay\s+(?P<init_spf>\d+)\s+msecs
-            \s*Minimum\s+hold\s+time\s+between\s+two\s+consecutive\s+SPFs\s+(?P<min_spf>\d+)\s+msecs
-            \s*Maximum\s+wait\s+time\s+between\s+two\s+consecutive\s+SPFs\s+(?P<max_spf>\d+)\s+msecs
+            \s*Minimum\s+hold\s+time\s+between\s+two\s+consecutive
+            \s+SPFs\s+(?P<min_spf>\d+)\s+msecs
+            \s*Maximum\s+wait\s+time\s+between\s+two\s+consecutive
+            \s+SPFs\s+(?P<max_spf>\d+)\s+msecs
         """
         regex = re.compile(process_pattern, re.VERBOSE + re.DOTALL)
         match = regex.search(text)

lint.sh → tests/lint.sh

@@ -14,7 +14,7 @@ echo "YAML linting started"
 for f in $(find . -name "*.yml"); do
   # Print the filename, then run 'yamllist' in strict mode
   echo "checking $f"
-  yamllint -s $f
+  yamllint --strict $f
   # Sum the rc from yamllint with the sum
   rc=$((rc + $?))
 done
@@ -25,7 +25,7 @@ echo "Python linting started"
 for f in $(find . -name "*.py"); do
   # Print the filename, then run 'pylint'
   echo "checking $f"
-  pylint $f
+  pylint --score n $f
   # Sum the rc from pylint with the sum
   rc=$((rc + $?))
 done

tests/test_ios_ospf_neighbor.yml

@@ -24,7 +24,7 @@
       - "data[0].state == 'full'"
       - "data[0].role == '-'"
       - "data[0].deadtime == '00:00:35'"
-      - "data[0].deadsec == 35"
+      - "data[0].deadtime_sec == 35"
       - "data[0].peer == '10.125.95.7'"
       - "data[0].intf == 'gigabiteth0/1'"
 
@@ -36,7 +36,7 @@
       - "data[1].state == 'full'"
       - "data[1].role == 'dr'"
       - "data[1].deadtime == '00:00:34'"
-      - "data[1].deadsec == 34"
+      - "data[1].deadtime_sec == 34"
       - "data[1].peer == '10.125.95.137'"
       - "data[1].intf == 'port-chan8.33'"
 
@@ -48,7 +48,7 @@
       - "data[2].state == '2way'"
       - "data[2].role == 'drother'"
       - "data[2].deadtime == '11:22:33'"
-      - "data[2].deadsec == 40953"
+      - "data[2].deadtime_sec == 40953"
       - "data[2].peer == '10.108.0.0'"
       - "data[2].intf == 'nve8'"
 ...