IntegrityError when running custom scripts inside branches

[lucagubler]

Plugin Version

0.4.0

NetBox Version

4.1

Python Version

3.10.12

Steps to Reproduce

1. Create a new branch and activate it.
2. Execute a custom script to reserve VRFs with RDs (Example script down below)
   • Script fails to create the second VRF
3. Switch back to the main branch and try to execute the script
   • Now it should work

This example script can be used to create VRFs and RDs.

from extras.scripts import *
from tenancy.models import Tenant
from ipam.models import RouteTarget, VRF


class ExampleCustomScript(Script):
    class Meta:
        name = "Example Custom Script"
        description = "Create NetBox VRFs"

    tenant = ObjectVar(
        model=Tenant, description="Tenant to generate cloud config for", label="Tenant"
    )

    def get_next_available_rd(self):
        """Returns the next available RD in the range 1001-1999."""
        existing_rds = {
            int(vrf.rd.split(":")[1]) for vrf in VRF.objects.exclude(rd__isnull=True)
        }
        for rd in range(1001, 1999):
            if rd not in existing_rds:
                return rd
        self.log_failure("No available RD in the range 1001-1999")
        sys.exit(1)

    def create_vrf_and_rt(self, tenant, tenant_name, use_case):
        """Creates a VRF and corresponding Route Target."""
        vrf_name = f"{tenant_name}-{use_case}"
        rd_number = self.get_next_available_rd()
        rd = f"12345:{rd_number}"

        vrf = VRF(name=vrf_name, rd=rd, tenant=tenant)
        vrf.save()
        self.log_success(f"Created VRF {vrf} with RD {rd}")

        route_target = RouteTarget(
            name=rd, tenant=tenant, description=f"{tenant_name} - {use_case}"
        )
        route_target.save()
        self.log_success(f"Created Route Target {route_target}")


    def run(self, data, commit):
        tenant = data["tenant"]
        tenant_name = tenant.slug

        for use_case in ["UC_A", "UC_B"]:
            # Create VRF and Route Target
            vrf = self.create_vrf_and_rt(tenant, tenant_name, use_case)

And here's the error I receive when running this custom script

An exception occurred: IntegrityError: duplicate key value violates unique constraint "ipam_vrf_rd_key" DETAIL: Key (rd)=(12345:1002) already exists.

Traceback (most recent call last):
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/netbox/venv/lib/python3.10/site-packages/psycopg/cursor.py", line 97, in execute
    raise ex.with_traceback(None)
psycopg.errors.UniqueViolation: duplicate key value violates unique constraint "ipam_vrf_rd_key"
DETAIL:  Key (rd)=(12345:1002) already exists.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/netbox/netbox/extras/jobs.py", line 45, in run_script
    script.output = script.run(data, commit)
  File "/opt/netbox/netbox/scripts/onway_cloud.py", line 444, in run
    vrf = self.create_vrf_and_rt(tenant, tenant_name, use_case)
  File "/opt/netbox/netbox/scripts/onway_cloud.py", line 410, in create_vrf_and_rt
    vrf.save()
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 822, in save
    self.save_base(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 909, in save_base
    updated = self._save_table(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 1071, in _save_table
    results = self._do_insert(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/base.py", line 1112, in _do_insert
    return manager._insert(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/query.py", line 1847, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1823, in execute_sql
    cursor.execute(sql, params)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/netbox/venv/lib/python3.10/site-packages/psycopg/cursor.py", line 97, in execute
    raise ex.with_traceback(None)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "ipam_vrf_rd_key"
DETAIL:  Key (rd)=(12345:1002) already exists.

Expected Behavior

All VRFs/RD should be created successfully.

Observed Behavior

The first VRF is created, but the second fails. It looks like the lookup, e.g. VRF.objects.all(), looks for data from the main branch. This works for the first iteration and a new VRF is created in the new branch. But for the second iteration, the script wants to reserve the same VRF/RD again because it's still available on the main branch. But then it fails because that RD is already taken in my new branch...

[arthanson]

Updated the script, it was missing all the imports. Noticed a potential other issue, after running I went to the main branch and bulk deleted the two created VRFs, then went to the test branch and tried to bulk-delete the single created VRF and got an error VRF matching query does not exist.

[arthanson]

The issue is the script params, the script only runs on the main branch as it's not branch aware, but when you add the script params it is pulling from the active branch so is causing inconsistencies. Am exploring options to fix