After upgrade from 3.5 to 3.7, instantiating a DeviceType which contains nested InventoryItems throws "InventoryItem matching query does not exist"

[padthaitofuhot]

Deployment Type

Self-hosted

NetBox Version

v3.7.4

Python Version

3.11

Steps to Reproduce

1. Upgrade from netbox-docker v3.5-2.6.1 to v3.7-2.8.0 via git release branch
2. Upgrade from Postgres 15 to Postgres 16 via sql dump/restore
3. Observe migrations complete successfully
4. Create a new DeviceType
5. Add InventoryItem A to new DeviceType
6. Add InventoryItem B to new DeviceType with InventoryItem A as Parent
7. Create a new Device from this DeviceType

Expected Behavior

A new Device is created with nested InventoryItems from DeviceType.

Observed Behavior

Instantiating a DeviceType which contains nested InventoryItems throws:

DoesNotExist at /dcim/devices/add/

InventoryItem matching query does not exist.

Request Method: 	POST
Request URL: 	http://netbox.mydomain.tld/dcim/devices/add/
Django Version: 	4.2.11
Exception Type: 	DoesNotExist
Exception Value: 	

InventoryItem matching query does not exist.

Exception Location: 	/opt/netbox/venv/lib/python3.11/site-packages/django/db/models/query.py, line 637, in get
Raised during: 	dcim.views.DeviceEditView
Python Executable: 	/opt/netbox/venv/bin/python
Python Version: 	3.11.4
Python Path: 	

['/opt/netbox/netbox/',
 '/usr/lib/python311.zip',
 '/usr/lib/python3.11',
 '/usr/lib/python3.11/lib-dynload',
 '/opt/netbox/venv/lib/python3.11/site-packages']

Server time: 	Tue, 02 Apr 2024 00:05:31 +0000

Environment:


Request Method: POST
Request URL: http://netbox.mydomain.tld/dcim/devices/add/

Django Version: 4.2.11
Python Version: 3.11.4
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.humanize',
 'django.forms',
 'corsheaders',
 'debug_toolbar',
 'graphiql_debug_toolbar',
 'django_filters',
 'django_tables2',
 'django_prometheus',
 'graphene_django',
 'mptt',
 'rest_framework',
 'social_django',
 'taggit',
 'timezone_field',
 'core',
 'account',
 'circuits',
 'dcim',
 'ipam',
 'extras',
 'tenancy',
 'users',
 'utilities',
 'virtualization',
 'vpn',
 'wireless',
 'django_rq',
 'drf_spectacular',
 'drf_spectacular_sidecar']
Installed Middleware:
['graphiql_debug_toolbar.middleware.DebugToolbarMiddleware',
 'django_prometheus.middleware.PrometheusBeforeMiddleware',
 'corsheaders.middleware.CorsMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'django.middleware.security.SecurityMiddleware',
 'netbox.middleware.RemoteUserMiddleware',
 'netbox.middleware.CoreMiddleware',
 'netbox.middleware.MaintenanceModeMiddleware',
 'django_prometheus.middleware.PrometheusAfterMiddleware']



Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/netbox/views/generic/object_views.py", line 175, in dispatch
    return super().dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/netbox/views/generic/base.py", line 26, in dispatch
    return super().dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/utilities/views.py", line 106, in dispatch
    return super().dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/views/generic/base.py", line 143, in dispatch
    return handler(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/netbox/views/generic/object_views.py", line 269, in post
    obj = form.save()
          ^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/forms/models.py", line 542, in save
    self.instance.save()
    ^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/dcim/models/devices.py", line 1054, in save
    self._instantiate_components(self.device_type.inventoryitemtemplates.all(), bulk_create=False)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/dcim/models/devices.py", line 999, in _instantiate_components
    components = [obj.instantiate(device=self) for obj in queryset]
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/dcim/models/devices.py", line 999, in <listcomp>
    components = [obj.instantiate(device=self) for obj in queryset]
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/netbox/dcim/models/device_component_templates.py", line 765, in instantiate
    parent = InventoryItem.objects.get(name=self.parent.name, **kwargs) if self.parent else None
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/models/query.py", line 637, in get
    raise self.model.DoesNotExist(
    ^

Exception Type: DoesNotExist at /dcim/devices/add/
Exception Value: InventoryItem matching query does not exist.

[padthaitofuhot]

This issue looks a lot like #12118 which was fixed in PR #12170.

[padthaitofuhot]

BTW - I opened this issue here (rather than in docker-netbox) because it does not seem like an issue with the Docker image or related tooling and rather more like an issue with schema or a script.

[jeremystretch]

Thank you for opening a bug report. I was unable to reproduce the reported behavior on NetBox v3.7.4. Please re-confirm the reported behavior on the current stable release and adjust your post above as necessary. Remember to provide detailed steps that someone else can follow using a clean installation of NetBox to reproduce the issue. Remember to include the steps taken to create any initial objects or other data.

[kkthxbye-code]

Replicates fine here following his steps. The offending commit is this:

32264ac

The cause is pretty obvious, inventory items are intentionally created with bulk_create=False to avoid trying to reference a non-existant object.

The above commit moves the instantiation of the inventory items outside the bulk_create check. Reverting the commit allows creating the device again.

[jeremystretch]

Thanks @kkthxbye-code, would you like to submit a PR for this?

[padthaitofuhot]

@jeremystretch @kkthxbye-code I got it. I'll have a PR shortly.