v3.5 reports & scripts cleanup

[jeremystretch]

Proposed Changes

Starting an issue to capture miscellaneous cleanup items relating to recent work around custom scripts and reports in preparation for the v3.5 release.

Justification

This issue will serve as a standing collector for miscellanea that don't quite warrant their own individual issue. Substantial work items should still have individual issues opened.

(Comments hidden below have been resolved in the 12081-cleanup branch.)

[arthanson]

Somewhat of a side issue, but the docs for running RQ should probably be improved, I'd suggest adding a section to "Installation and Upgrade" for running "python netbox/manage.py rqworker", there is only one part in the documents I could find under "Plugins -> Developing Plugins -> Background Tasks" a note on "Configuring the RQ worker process" where it shows using the management command. The RQ docs just show "rq worker --with-scheduler" which is probably not what you want.

[arthanson]

I changed the device script and re-uploaded it and got the following error - was from having it the same filename but it should probably display a error modal instead of crash.:

Traceback (most recent call last):
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 56, in inner
    response = get_response(request)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 103, in view
    return self.dispatch(request, *args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/netbox/netbox/views/generic/object_views.py", line 167, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/netbox/netbox/views/generic/base.py", line 26, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/netbox/utilities/views.py", line 99, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 142, in dispatch
    return handler(request, *args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/netbox/netbox/views/generic/object_views.py", line 261, in post
    obj = form.save()
  File "/Users/ahanson/dev/work/netbox/netbox/core/forms/model_forms.py", line 111, in save
    return super().save(*args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/forms/models.py", line 548, in save
    self.instance.save()
  File "/Users/ahanson/dev/work/netbox/netbox/extras/models/reports.py", line 68, in save
    return super().save(*args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 812, in save
    self.save_base(
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 863, in save_base
    updated = self._save_table(
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 1006, in _save_table
    results = self._do_insert(
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 1047, in _do_insert
    return manager._insert(
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/query.py", line 1791, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1660, in execute_sql
    cursor.execute(sql, params)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/debug_toolbar/panels/sql/tracking.py", line 230, in execute
    return self._record(self.cursor.execute, sql, params)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/debug_toolbar/panels/sql/tracking.py", line 154, in _record
    return method(sql, params)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 102, in execute
    return super().execute(sql, params)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/Users/ahanson/dev/work/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "core_managedfile_unique_root_path"
DETAIL:  Key (file_root, file_path)=(reports, devices.py) already exists.

[jeremystretch]

When creating a report/script from a DataFile, the path of the origin file is not replicated. For example, a DataFile with a path of foo/bar/myreport.py will create a file myreport.py in the reports root.

[jeremystretch]

@sleepinggenius2 this issue is only for recording observed bugs & limitations in ongoing work. Please hold such discussion until the beta release has been made available for testing.

[sleepinggenius2]

@sleepinggenius2 this issue is only for recording observed bugs & limitations in ongoing work. Please hold such discussion until the beta release has been made available for testing.

I apologize. I read through #12059 and it wasn't clear where this was being accounted for, so I thought it was a relevant question to ask in this context with respect to a potential limitation. I will wait for the beta release for further comments.

[jeremystretch]

@kkthxbye-code has provided an example repo with a few scripts to help test upgrading & loading. There are currently two issues:

1. Relative imports from within a module don't work. We likely just need to tweak the module loading logic.
2. Empty modules get replicated as database objects. This isn't necessarily a problem, however it can be confusing. We'll need to figure out the ideal strategy for managing these files.