Meta: Unit test coverage for netbox_manager/

[berendt]

Background

Analogous to osism/container-image-inventory-reconciler#524 and osism/python-osism#2199. Once test infrastructure (pytest harness, a Zuul netbox-manager-unit-tests job) has landed as a separate foundation PR, actual unit test coverage has to be added function group by function group. This meta issue tracks the overall rollout plan so nothing gets lost and links all sub-issues.

The package today has no unit tests at all — Zuul check/periodic-daily only run flake8, mypy, python-black, yamllint (see .zuul.yaml). The package contains ~4100 lines of Python — netbox_manager/main.py (~3236 LOC, Typer CLI + all helper functions) and netbox_manager/dtl.py (~868 LOC, Device Type Library port creators) — much of which is pure transformation logic that can be exercised without a live NetBox.

Approach: start with pure-logic helpers (fast feedback, no mocking), then move to task / playbook builders, filesystem helpers, autoconf generation, validation, DTL, and finally a CLI wiring smoke test. Each sub-issue is self-contained and reviewable on its own.

Progress

Tier 0 — Foundation (prerequisite)

• Introduce foundation for unit tests (pytest + Zuul CI integration) #233 — pytest harness + tests/ tree at repo root, test-requirements.txt, shared fixtures in conftest.py (mock pynetbox device/interface/IP/VRF/cluster/config-context/prefix shapes, dynaconf settings stub, mocked ansible_runner), new Zuul netbox-manager-unit-tests job in check / periodic-daily

Tier 1 — Pure logic (quick wins, no NetBox stubs)

• Unit tests: Tier 1 — pure-logic helpers in netbox_manager/main.py #247 — pure-logic helpers in netbox_manager/main.py (the five function groups below)

• netbox_manager/main.py — settings/role helpers: get_node_roles (main.py:122), get_switch_roles (main.py:127) — settings override vs _DEFAULT_NODE_ROLES / _DEFAULT_SWITCH_ROLES fallback

• netbox_manager/main.py — generic helpers: deep_merge (main.py:195, nested dicts, list-override, deepcopy isolation, conflicting types), get_leading_number (main.py:170), get_resource_name (main.py:236, name → address → id → "unknown"), get_device_role_slug (main.py:224, slug vs name vs missing role), find_device_names_in_structure (main.py:176, recursion through dict / list), is_virtual_interface (main.py:2051, type.value vs type.label vs missing), _split_tasks_by_type (main.py:1995)

• netbox_manager/main.py — task-filter helpers: should_skip_task_by_filter (main.py:289, -/_ normalisation), extract_device_names_from_task (main.py:296, device:<str> field, device-creation name, nested), should_skip_task_by_device_filter (main.py:314, empty list, substring vs no-match)

• netbox_manager/main.py — autoconf helpers: get_autoconf_prefix (main.py:2280, 200-aa → 299-autoconf, 1000-xx → 1099-autoconf), _extract_device_names_from_autoconf_task (main.py:2292), _filter_tasks_by_device / _filter_tasks_by_type_by_device / _filter_portchannel_tasks_by_device (main.py:2307–2341)

• netbox_manager/main.py — loopback gate helpers: has_sonic_hwsku_parameter (main.py:1266, missing custom_fields / non-dict / missing hwsku), should_have_loopback_interface (main.py:1277, node-role pass-through, switch-role + hwsku gate, switch-by-device-type-name fallback)

Tier 2 — Task / playbook builders (settings-aware, no NetBox calls)

• Unit tests: Tier 2 — task / playbook builders in netbox_manager/main.py #249 — task / playbook builders in netbox_manager/main.py (the three function groups below)

• netbox_manager/main.py — create_netbox_task (main.py:328, default state, update_vc_child extraction for device_interface, register / ignore_errors, settings-driven netbox_url / netbox_token / validate_certs)

• netbox_manager/main.py — create_uri_task (main.py:373, /api/ + api/ prefix stripping, leading-slash normalisation, body-empty branch removes body/body_format, status-code list, register / ignore_errors)

• netbox_manager/main.py — create_ansible_playbook (main.py:442, Jinja render of vars + tasks, basename in title), ProperIndentDumper (main.py:39, nested-sequence indentation round-trip)

Tier 3 — Filesystem helpers (tmp_path)

• Unit tests: Tier 3 — filesystem helpers in netbox_manager/main.py #259 — filesystem helpers in netbox_manager/main.py (the three function groups below)

• netbox_manager/main.py — find_yaml_files (main.py:208, .yml + .yaml, sorted), load_global_vars (main.py:249, missing VARS, missing dir, multi-file deep-merge order, malformed YAML logged-not-raised, empty file)

• netbox_manager/main.py — discover_resource_files (main.py:677, top-level + numbered subdirs, vars dir excluded, --limit at file and directory level, missing dir handled), detect_site_folders (main.py:2229, only \d+-… pattern, sorted), extract_device_names_from_folder (main.py:2249, ignores non-list / non-dict, dedup)

• netbox_manager/main.py — _write_autoconf_files (main.py:2011, per-resource-type filename with _→-, makedirs, settings.RESOURCES vs explicit dir, return count)

Tier 4 — handle_file orchestration (mock ansible_runner)

• Unit tests: Tier 4 — handle_file orchestration in netbox_manager/main.py #252 — handle_file orchestration in netbox_manager/main.py (the function group below)

• netbox_manager/main.py — handle_file (main.py:456): YAML load errors (file missing, malformed YAML, non-list root, non-dict task, empty task), per-task branches (vars deep-merge, debug, uri, default → create_netbox_task), task / device filter pass-through, register extraction, ignore_errors propagation, show_playbooks short-circuit, dry-run skip vs ansible_runner.run invocation, fail_fast raises typer.Exit

Tier 5 — Loopback + cluster + interface-label + portchannel generation (mock pynetbox)

• Unit tests: Tier 5 — loopback / cluster / interface-label / portchannel generation in netbox_manager/main.py #253 — loopback / cluster / interface-label / portchannel generators in netbox_manager/main.py (the three function groups below)

• netbox_manager/main.py — _generate_loopback_interfaces (main.py:1320), group_devices_by_cluster (main.py:1414), calculate_loopback_ips (main.py:1427, missing/invalid position, byte_4 arithmetic, IPv4-only vs IPv4+IPv6, network arithmetic + IPv6 suffix mapping)

• netbox_manager/main.py — _get_cluster_segment_config_context (main.py:1347, name-match → data, name-match + empty data → {}, no match → {}, exception → {}), _generate_cluster_loopback_tasks (main.py:1489)

• netbox_manager/main.py — _generate_device_interface_labels (main.py:1627), _generate_portchannel_tasks (main.py:1747) — split further if needed

Tier 6 — Autoconf generation + dispatcher (mock pynetbox)

• Unit tests: Tier 6 — autoconf generation + dispatcher in netbox_manager/main.py #254 — autoconf generation + dispatcher in netbox_manager/main.py (the two function groups below)

• netbox_manager/main.py — collect_interface_assignments (main.py:2067, virtual skipped, mac_address vs mac_addresses[0].mac_address fallback, devices sorted by name), collect_ip_assignments_by_interface (main.py:2111, OOB vs Primary path, IPv4 vs IPv6 split), _generate_autoconf_tasks (main.py:2161, switch vs non-switch split, OOB+primary merge by device name)

• netbox_manager/main.py — _run_autoconf_for_devices (main.py:2341, label/non-label device_interface merge, device-filter application across all sources, dry-run path, file-write path)

Tier 7 — Validation (mock pynetbox)

• Unit tests: Tier 7 — validation helpers in netbox_manager/main.py #255 — validation helpers in netbox_manager/main.py (the two function groups below)

• netbox_manager/main.py — validate_ip_addresses_have_prefixes (main.py:2835, valid IP w/ matching prefix, invalid IP-format → orphaned with reason, no matching prefix → orphaned, VRF vs global query branch, pynetbox.RequestError propagation)

• netbox_manager/main.py — validate_vrf_consistency (main.py:2946, IP w/o assigned object skipped, VM interface skipped, matching VRF passes, mismatch recorded, interface lookup error swallowed)

Tier 8 — Archive + connection helpers

• Unit tests: Tier 8 — archive + connection helpers in netbox_manager/main.py #256 — archive + connection helpers in netbox_manager/main.py (the two function groups below)

• netbox_manager/main.py — export_archive (main.py:1091, missing libraries → exit, git commit info (with InvalidGitRepositoryError handled), tar contents include COMMIT_INFO.txt + dirs, --image non-Linux exit, --image happy path with mocked subprocess); import_archive (main.py:1200, missing input → exit, rsync invocation per dir, rsync failure → exit)

• netbox_manager/main.py — validate_netbox_connection (main.py:132, success, pynetbox.RequestError, generic exception); create_netbox_api (main.py:216, SSL-ignore branch sets verify=False); process_device_and_module_types (main.py:650, devicetypes vs moduletypes branches, missing library skip)

Tier 9 — DTL (dtl.py)

• Unit tests: Tier 9 — Device Type Library importer (netbox_manager/dtl.py) #257 — Device Type Library importer in netbox_manager/dtl.py (the five function groups below)

• netbox_manager/dtl.py — Repo: slug_format (dtl.py:89), get_devices (dtl.py:92, vendor filter, file walking), parse_files (dtl.py:121, slug filter, YAML round-trip)

• netbox_manager/dtl.py — NetBox (mock pynetbox): connect_api (dtl.py:174, SSL-ignore branch), verify_compatibility (dtl.py:192), get_manufacturers (dtl.py:201), create_manufacturers (dtl.py:204, idempotent — only missing vendors created), create_device_types (dtl.py:234), create_module_types (dtl.py:313)

• netbox_manager/dtl.py — DeviceTypes lookup helpers: get_device_types / get_power_ports / get_rear_ports / get_module_power_ports / get_module_rear_ports (dtl.py:386–421), get_device_type_ports_to_create / get_module_type_ports_to_create (dtl.py:421–439, dedup against existing)

• netbox_manager/dtl.py — DeviceTypes device port creators (dtl.py:439–663): create_interfaces, create_power_ports, create_console_ports, create_power_outlets, create_console_server_ports, create_rear_ports, create_front_ports, create_device_bays, create_module_bays — common patterns: existing-port skip, batch create, LogHandler.log_device_ports_created invocation

• netbox_manager/dtl.py — DeviceTypes module port creators (dtl.py:666–845): create_module_interfaces, create_module_power_ports, create_module_console_ports, create_module_power_outlets, create_module_console_server_ports, create_module_rear_ports, create_module_front_ports; plus upload_images (dtl.py:847, base-URL handling, token header, image POST per file)

Tier 10 — CLI wiring (smoke tests)

• Unit tests: Tier 10 — CLI command wiring (smoke tests) in netbox_manager/main.py #258 — CLI command wiring (smoke tests) in netbox_manager/main.py (the Typer commands below)

• netbox_manager/main.py — Typer commands with _run_main, _run_autoconf_for_devices, validate_*, archive helpers patched out: run_command (main.py:1012), autoconf_command (main.py:2485), validate_command (main.py:3046), purge_command (main.py:2598, dry-run only — destructive in real mode), version_command (main.py:3217), main (main.py:3230); also init_logger / signal_handler_sigint (main.py:631–647) and callback_version (main.py:726)

Conventions for sub-issues

• Scope: one function group or one cohesive helper bundle per issue
• Each sub-issue lists test targets with file:line references
• Each sub-issue documents required mocks (pynetbox device / interface / IP / VRF / cluster / config-context / prefix shapes; ansible_runner.run; filesystem via tmp_path; dynaconf settings stub; git.Repo; subprocess.run / check_call)
• Definition of Done: tests pass locally with pytest, the new Zuul netbox-manager-unit-tests job is green, flake8 / mypy / python-black remain green
• Depends on: Tier 0 foundation (separate PR)

Out of scope here

• Integration / end-to-end tests against a live NetBox (Ansible playbook execution path is not exercised here — covered by the integration testbed)
• Coverage thresholds / gates (separate follow-up)
• Python-version matrix in CI (separate follow-up)
• Tests for example/ resource files