Custom scripts are significantly slower after upgrading from 3.3.7 to 3.4.2

[desnoe]

NetBox version

v3.4.2

Python version

3.10

Steps to Reproduce

1. Install the below custom script that creates 100 devices on NetBox 3.3.7. On my machine, I get this execution message: Initiated: 2023-01-04 16:36 Duration: 0 minutes, 4.17 seconds Completed
2. Use the same script on NetBox 3.4.2. On my same machine, I now get: Started: 2023-01-04 16:37 Duration: 0 minutes, 59.31 seconds Completed

class Perf(Script):
    class Meta:
        name = "Device creation performance test"
        description = "Device creation performance test"

    def run(self, data, commit):
        for i in range(100):
            netbox_device = Device(
                name="device" + str(i),
                device_type=DeviceType.objects.first(),
                device_role=DeviceRole.objects.first(),
                site=Site.objects.first(),
            )
            netbox_device.save()

NB: this script is not optimised, but this is not the point. The point is that the same script have inconsistent execution times between versions.

Expected Behavior

The execution time should remain consistent between this 2 versions. Some dozens of percent of increase could still be acceptable.

Observed Behavior

The execution time for the reference script is ~14x times slower on version 3.4.2 compared to 3.3.7.

[kkthxbye-code]

Hi, this is not a bug and it has nothing to do with scripts. The worse performance specifically relates to creating devices after we fixed an uninteded bug where changelog entries were not created for device components.

#10694
#10900

Before when instantiating a device, components were created with bulk_create which doesn't send any signals, thus skipping both changelog generation and search indexing. The new approach still bulk creates most of the components, but sends the post_save signal manually after creation. As a result device instantiation is vastly slower than it were before. The superior performance was only possible because the old approach was incorrect.

There's an open issue considering moving both changelog creation and search indexing to background tasks: #11202

Do note that this would not eliminate the processing, it would just be moved to another process. The tradeof is a small delay before the changelog for the objects would be created and search indexing would finish.