Netbox virtual interfaces do not support tag filter

[inetman28]

Environment

• Python version: 3.6.8
• NetBox version: 2.6.7 and 2.6.11
• vCenter version: 6.5.0 and 6.7.0
• vCenter NetBox Sync version: 1.0.0

Steps to Reproduce

1. set up settings.py with two vcenter instances
2. python3 run.py -v

Expected Behavior

I expected that all vcenter objects will sync with netbox

Observed Behavior

I have two vcenter servers: vcsmgmt and vcsres-1. When object from vcsres-1 have been synced with netbox, second vcenter should be synced too but during sync second vcenter server remove virtual interfaces and ip address that was synced from first vcenter. See below log for example:

2020-01-14 13:58:10,214 [INFO] Comparing existing NetBox virtual_interfaces objects to current vCenter objects for pruning eligibility.
2020-01-14 13:58:10,214 [DEBUG] Sending GET to 'http://192.168.253.131:32769/api/virtualization/interfaces/?tag=vcsmgmt'

as you can see we get objects from netbox with 'vcsmgmt' tag

2020-01-14 13:58:10,340 [DEBUG] NetBox GET request OK; returned 200 status.
2020-01-14 13:58:10,341 [DEBUG] NetBox returned more than 50 objects. Sending GET to http://192.168.253.131:32769/api/virtualization/interfaces/?limit=50&offset=50&tag=vcsmgmt for additional objects.
2020-01-14 13:58:10,472 [DEBUG] NetBox returned more than 50 objects. Sending GET to http://192.168.253.131:32769/api/virtualization/interfaces/?limit=50&offset=100&tag=vcsmgmt for additional objects.
2020-01-14 13:58:10,595 [DEBUG] NetBox returned more than 50 objects. Sending GET to http://192.168.253.131:32769/api/virtualization/interfaces/?limit=50&offset=150&tag=vcsmgmt for additional objects.
2020-01-14 13:58:10,722 [DEBUG] NetBox returned more than 50 objects. Sending GET to http://192.168.253.131:32769/api/virtualization/interfaces/?limit=50&offset=200&tag=vcsmgmt for additional objects.
2020-01-14 13:58:10,810 [INFO] Comparison completed. 132 virtual_interfaces orphaned NetBox objects did not match.

but vcenter-netbox-sync will remove another objects without tag 'vcsmgmt' ('vcsres-1' for example)

2020-01-14 13:58:10,811 [DEBUG] The following objects did not match: [{'id': 85, 'virtual_machine': {'id': 15, 'url': 'http://192.168.253.131:32769/api/virtualization/virtual-machines/15/', 'name': 'cvp-2018.2.5_test'}, 'name': 'vNIC2', 'type': {'value': 0, 'label': 'Virtual'}, 'enabled': True, 'mtu': None, 'mac_address': '00:50:56:9D:63:AA', 'description': '', 'mode': None, 'untagged_vlan': None, 'tagged_vlans': [], 'tags': **['vCenter', 'Synced', 'vcsres-1**']}, {'id': 93, 'virtual_machine': {'id': 30, 'url': 'http://192.168.253.131:32769/api/virtualization/virtual-machines/30/', 'name': 'ISE-NODE1.ISE.TEST'}, 'name': 'vNIC2', 'type': {'value': 0, 'label': 'Virtual'}, 'enabled': True, 'mtu': None, 'mac_address': '00:50:56:9D:0E:CC', 'description': '', 'mode': None, 'untagged_vlan': None, 'tagged_vlans': [], 'tags': ['vCenter', 'Synced', 'vcsres-1']}, {'id': 102, 'virtual_machine': {'id': 31, 'url': 'http://192.168.253.131:32769/api/virtualization/virtual-machines/31/', 'name': 'EDGE5-RVOLODIN-TEST-0'}, 'name': 'vNIC2', 'type': {'value': 0, 'label': 'Virtual'}, 'enabled': False, 'mtu': None, 'mac_address': '00:50:56:9D:27:42', 'description': '', 'mode': None, 'untagged_vlan': None, 'tagged_vlans': [], 'tags': ['vCenter', 'Synced', 'vcsres-1']}, 

[synackray]

Thanks for the report @inetman28! I suspect the issue is related to the way the tags are being created. Either the tag has a name or slug value that not conforming to NetBox. When searching objects by tag NetBox is returning all items instead of the items that would otherwise contain the tag.

Would you mind sharing the tag name and slug for me? The output of the following URL would be very helpful in troubleshooting as well. Thank you!

http://192.168.253.131:32769/api/extras/tags/

[inetman28]

hi
yes, I can

HTTP 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

{
    "count": 6,
    "next": null,
    "previous": null,
    "results": [
        {
            "id": 1,
            "name": "Synced",
            "slug": "synced",
            "color": "9e9e9e",
            "comments": "",
            "tagged_items": 781
        },
        {
            "id": 2,
            "name": "vCenter",
            "slug": "vcenter",
            "color": "9e9e9e",
            "comments": "",
            "tagged_items": 781
        },
        {
            "id": 3,
            "name": "Orphaned",
            "slug": "orphaned",
            "color": "607d8b",
            "comments": "This applies to objects that have become orphaned. The source system which has previously provided the object no longer states it exists. An object with the 'Orphaned' tag will remain in this state until it ages out and is automatically removed.",
            "tagged_items": 0
        },
        {
            "id": 4,
            "name": "vcsmgmt",
            "slug": "vcsmgmt",
            "color": "9e9e9e",
            "comments": "",
            "tagged_items": 23
        },
        {
            "id": 5,
            "name": "vcsres-1",
            "slug": "vcsres-1",
            "color": "9e9e9e",
            "comments": "",
            "tagged_items": 451
        },
        {
            "id": 6,
            "name": "cdc-vc-01",
            "slug": "cdc-vc-01",
            "color": "9e9e9e",
            "comments": "",
            "tagged_items": 308
        }
    ]
}

[synackray]

Thank you! You've presented quite an interesting situation. Reviewing the NetBox API docs I see we can assign tags to /virtualization/interfaces/ via POST/PUT/PUSH methods but the GET method does not support the tag filter. I've documented this for NetBox v2.6.11 with the below screenshots.

I will take a bit of time to review this and see what alternative methods we have of comparing and syncing virtual machine interfaces. Great find!

NetBox Virtual Intefaces POST

NetBox Virtual Intefaces GET

[synackray]

I've submitted a patch for this issue. It works around the current limitation by collecting all interfaces and then filtering by tag post collection. I will check with the upstream NetBox community project to determine if it's the desired ability to tag and object without being able to query those tags as well.

[synackray]

I've applied the patch in develop and will run it through full testing before merging into master. Thanks again for the report!