Direct upgrade from 3.7.8 to 4.1.0: Migrations fail

[peteeckel]

Deployment Type

Self-hosted

NetBox Version

v4.1.0

Python Version

3.11

Steps to Reproduce

1. Start with a working 3.7.8 instance
2. Upgrade the /opt/netbox directory to 4.1.0 (master)
3. Run /opt/netbox/netbox/manage.py migrate

Expected Behavior

According to the documentation, an upgrade from the latest 3.x-Release (in this case, 3.7.8) to any 4.x-Release should work.

Observed Behavior

It does not.

(netbox) [root@dns netbox]# /opt/netbox/netbox/manage.py migrate
Operations to perform:
  Apply all migrations: account, auth, circuits, contenttypes, core, dcim, django_rq, extras, ipam, netbox_dns, sessions, social_django, taggit, tenancy, users, virtualization, vpn, wireless
Running migrations:
  Applying extras.0108_convert_reports_to_scripts... OK
  Applying extras.0109_script_model... OK
  Applying extras.0110_remove_eventrule_action_parameters... OK
  Applying extras.0111_rename_content_types... OK
  Applying tenancy.0015_contactassignment_rename_content_type... OK
  Applying users.0005_alter_user_table...Traceback (most recent call last):
  File "/opt/netbox/netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/lib64/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/lib64/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/netbox/lib64/python3.11/site-packages/django/core/management/base.py", line 413, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/netbox/lib64/python3.11/site-packages/django/core/management/base.py", line 459, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/lib64/python3.11/site-packages/django/core/management/base.py", line 107, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/lib64/python3.11/site-packages/django/core/management/commands/migrate.py", line 356, in handle
    post_migrate_state = executor.migrate(
                         ^^^^^^^^^^^^^^^^^
  File "/opt/netbox/lib64/python3.11/site-packages/django/db/migrations/executor.py", line 135, in migrate
    state = self._migrate_all_forwards(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/lib64/python3.11/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
    state = self.apply_migration(
            ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/lib64/python3.11/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
    state = migration.apply(state, schema_editor)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/lib64/python3.11/site-packages/django/db/migrations/migration.py", line 132, in apply
    operation.database_forwards(
  File "/opt/netbox/lib64/python3.11/site-packages/django/db/migrations/operations/special.py", line 193, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/opt/netbox/netbox/users/migrations/0005_alter_user_table.py", line 17, in update_content_types
    CustomField.objects.filter(related_object_type_id=netboxuser_ct.id).update(related_object_type_id=user_ct.id)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/lib64/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/lib64/python3.11/site-packages/django/db/models/query.py", line 1476, in filter
    return self._filter_or_exclude(False, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/lib64/python3.11/site-packages/django/db/models/query.py", line 1494, in _filter_or_exclude
    clone._filter_or_exclude_inplace(negate, args, kwargs)
  File "/opt/netbox/lib64/python3.11/site-packages/django/db/models/query.py", line 1501, in _filter_or_exclude_inplace
    self._query.add_q(Q(*args, **kwargs))
  File "/opt/netbox/lib64/python3.11/site-packages/django/db/models/sql/query.py", line 1613, in add_q
    clause, _ = self._add_q(q_object, self.used_aliases)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/lib64/python3.11/site-packages/django/db/models/sql/query.py", line 1645, in _add_q
    child_clause, needed_inner = self.build_filter(
                                 ^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/lib64/python3.11/site-packages/django/db/models/sql/query.py", line 1495, in build_filter
    lookups, parts, reffed_expression = self.solve_lookup_type(arg, summarize)
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/lib64/python3.11/site-packages/django/db/models/sql/query.py", line 1307, in solve_lookup_type
    _, field, _, lookup_parts = self.names_to_path(lookup_splitted, self.get_meta())
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/lib64/python3.11/site-packages/django/db/models/sql/query.py", line 1772, in names_to_path
    raise FieldError(
django.core.exceptions.FieldError: Cannot resolve keyword 'related_object_type_id' into field. Choices are: choice_set, choice_set_id, created, default, description, filter_logic, group_name, id, is_cloneable, label, last_updated, name, object_type, object_type_id, object_types, required, search_weight, type, ui_editable, ui_visible, validation_maximum, validation_minimum, validation_regex, weight

With an intermediate step to 4.0.11, the migration works:

(netbox) [root@dns netbox]# /opt/netbox/netbox/manage.py migrate
Operations to perform:
  Apply all migrations: account, auth, circuits, contenttypes, core, dcim, django_rq, extras, ipam, netbox_dns, sessions, social_django, taggit, tenancy, users, virtualization, vpn, wireless
Running migrations:
  Applying dcim.0186_location_facility... OK
  Applying dcim.0187_alter_device_vc_position... OK
  Applying extras.0112_tag_update_object_types... OK
  Applying extras.0113_customfield_rename_object_type... OK
  Applying extras.0114_customfield_add_comments... OK
  Applying extras.0115_convert_dashboard_widgets... OK
  Applying extras.0116_custom_link_button_color... OK
  Applying netbox_dns.0002_contact_description_registrar_description... OK
  Applying netbox_dns.0003_default_view... OK
  Applying netbox_dns.0004_create_and_assign_default_view... OK
  Applying netbox_dns.0005_alter_zone_view_not_null... OK
  Applying netbox_dns.0006_templating... OK
  Applying netbox_dns.0007_alter_ordering_options... OK
  Applying netbox_dns.0008_view_prefixes... OK
  Applying netbox_dns.0009_rename_contact_registrationcontact... OK
  Applying social_django.0016_alter_usersocialauth_extra_data... OK
  Applying users.0005_alter_user_table... OK
  Applying users.0006_custom_group_model... OK
  Applying users.0007_objectpermission_update_object_types... OK
  Applying users.0008_flip_objectpermission_assignments... OK
  Applying users.0009_update_group_perms... OK
  Applying vpn.0005_rename_indexes... OK
(netbox) [root@dns netbox]# git checkout master 
Previous HEAD position was 0b120e6ad Merge pull request #17346 from netbox-community/develop
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
(netbox) [root@dns netbox]# /opt/netbox/netbox/manage.py migrate
Operations to perform:
  Apply all migrations: account, auth, circuits, contenttypes, core, dcim, django_rq, extras, ipam, netbox_dns, sessions, social_django, taggit, tenancy, users, virtualization, vpn, wireless
Running migrations:
  Applying core.0011_move_objectchange... OK
  Applying extras.0117_move_objectchange... OK
  Applying extras.0118_customfield_uniqueness... OK
  Applying extras.0119_notifications... OK
  Applying circuits.0044_circuit_groups... OK
  Applying core.0012_job_object_type_optional... OK
  Applying extras.0120_eventrule_event_types... OK
  Applying extras.0121_customfield_related_object_filter... OK
  Applying dcim.0188_racktype... OK
  Applying dcim.0189_moduletype_rack_airflow... OK
  Applying dcim.0190_nested_modules... OK
  Applying ipam.0070_vlangroup_vlan_id_ranges... OK
  Applying virtualization.0039_virtualmachine_serial_number... OK
  Applying virtualization.0040_convert_disk_size... OK
  Applying wireless.0009_wirelesslink_distance... OK

I am not very surprised that migrating from 3.7.8 to 4.1.0 does not work. The documentation does seem to lead in that direction, and IMHO this is a documentation bug.

[jeremystretch]

Just to confirm, upgrading from v3.7.8 to v4.1.0 is supported and should work as expected.

[peteeckel]

OK, so it's not a documentation bug but a real one ...