Skip to content

chore: update CI workflows, add pytest-timeout, and switch to ThreadPoolExecutor#1

Merged
marcinpsk merged 23 commits intomainfrom
develop
Dec 15, 2025
Merged

chore: update CI workflows, add pytest-timeout, and switch to ThreadPoolExecutor#1
marcinpsk merged 23 commits intomainfrom
develop

Conversation

@marcinpsk
Copy link
Owner

@marcinpsk marcinpsk commented Dec 14, 2025

Summary by CodeRabbit

  • New Features

    • Git URL validation for repository inputs.
    • Example environment file with NetBox integration placeholders.
  • Bug Fixes

    • Safer image upload handling to avoid resource leaks.
    • Clearer error messages for invalid Git URLs and missing env vars.
    • Skipped front/rear port entries are now logged when unresolved.
  • Improvements

    • Container runs as a non-root user.
    • Test timeout configured for faster failure detection.
    • CI workflow uses a dynamic repository-based image reference and improved permissions.
  • Chores

    • Dependabot and CI workflow configuration tweaks.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link

coderabbitai bot commented Dec 14, 2025

Warning

Rate limit exceeded

@marcinpsk has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 24 minutes and 33 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 287bfb1 and da2859c.

📒 Files selected for processing (2)
  • log_handler.py (1 hunks)
  • netbox_api.py (6 hunks)

Walkthrough

Validate Git URLs before cloning, add single-file YAML parsing and switch parsing to threads; improve NetBox port resolution and image upload safety; add CI/Dependabot updates and non-root Docker runtime; add test-timeout, env example, and expanded logging/error messages.

Changes

Cohort / File(s) Summary
CI/CD & Dependabot
\.github/workflows/docker.yml, \.github/dependabot.yml
Workflow permissions added; repo name lowercased into REPO_LOWER env and GHCR image ref made dynamic; dependabot pip/github-actions entries normalized and schedule set to weekly.
Container
Dockerfile
Switch final runtime to non‑root appuser (create UID 1000, chown /app, switch USER).
Project & Testing Config
pyproject.toml, env.example
Add pytest-timeout>=2.4.0 and [tool.pytest.ini_options] timeout settings; add env.example with NetBox/integration and repo variables.
Repository handling & parsing
repo.py
Add validate_git_url(url) and parse_single_file(file); validate remote URL before cloning when repo path is missing; change parse_files to ThreadPoolExecutor; add/expand docstrings.
NetBox API & uploads
netbox_api.py
Resolve rear_port names to IDs and drop/log unresolved front-ports and module-fronts; rework upload_images() to open files per-request, PATCH with file handles, increment counters, and ensure handles closed in finally; add/expand docstrings.
Logging & CLI
log_handler.py, nb-dt-import.py
Remove custom __new__ in LogHandler, add InvalidGitURL mapping and conditional stack-trace printing when verbose; refine missing-env messaging and add docstring to main().

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

  • Review focus:
    • netbox_api.py — ensure all file handles are reliably closed on success and on all error/exception paths; validate behavior on partial upload failures.
    • repo.py — verify validate_git_url() covers edge-case Git URL formats and that ThreadPoolExecutor is safe with the YAML loader used.
    • Dockerfile — confirm UID 1000 and chown interactions with potential host-mounted volumes.
    • .github/workflows/docker.yml — confirm dynamic GHCR image name and permissions are correct for registry writes.

Poem

🐇 I sniffed each URL with careful delight,
Threads now chatter softly through the night,
Files closed tidy after every send,
Ports find their partners — mismatches end,
A happy hop — imports run polite!

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 77.27% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main changes: CI workflow updates, pytest-timeout addition, and ThreadPoolExecutor switch. It covers the primary modifications across multiple files.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between fafc0ab and 82a2f17.

📒 Files selected for processing (2)
  • .github/workflows/docker.yml (2 hunks)
  • .github/workflows/tests.yml (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: test
🔇 Additional comments (1)
.github/workflows/docker.yml (1)

53-60: labels: ${{ steps.meta.outputs.labels }} wiring looks correct

Good alignment with docker/metadata-action outputs; helps with traceability in the registry.

@marcinpsk marcinpsk changed the title chore: update CI workflow chore: update Docker image reference and test output in CI workflows Dec 14, 2025
@marcinpsk
Copy link
Owner Author

@coderabbitai Check for security issues and inconsistencies across the repo - this repo wasn't reviewed yet.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 82a2f17 and 9c72fa4.

⛔ Files ignored due to path filters (1)
  • uv.lock is excluded by !**/*.lock
📒 Files selected for processing (2)
  • pyproject.toml (2 hunks)
  • repo.py (1 hunks)
🔇 Additional comments (2)
pyproject.toml (2)

31-33: LGTM! Timeout configuration aligns with threading model.

The thread-based timeout method is appropriate and aligns well with the switch to ThreadPoolExecutor in repo.py. The 20-second timeout is reasonable for preventing test hangs.


21-21: pytest-timeout>=2.4.0 is valid and points to the current stable release.

Version 2.4.0 exists on PyPI and is the latest stable release, making the dependency constraint appropriate for preventing hanging tests.

@coderabbitai
Copy link

coderabbitai bot commented Dec 14, 2025

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash

echo "=== Repository Structure ==="
fd -t f -d 3 | head -50

echo -e "\n=== Checking for sensitive files ==="
fd -t f -e env -e key -e pem -e p12 -e pfx -e crt -e cer

echo -e "\n=== Checking workflow files ==="
cat .github/workflows/docker.yml
echo -e "\n---"
cat .github/workflows/tests.yml

echo -e "\n=== Checking for hardcoded secrets patterns ==="
rg -i -n "password|secret|api[_-]?key|token|credentials|private[_-]?key" --type yaml --type json --type py -g '!*.md' -g '!*.txt' | head -30

echo -e "\n=== Checking Python dependencies ==="
fd -t f requirements.txt -x cat {}
fd -t f setup.py -x cat {}
fd -t f pyproject.toml -x cat {}

echo -e "\n=== Main Python files ==="
fd -t f -e py | head -20

Length of output: 4255


🏁 Script executed:

#!/bin/bash

echo "=== Dockerfile Review ==="
cat Dockerfile

echo -e "\n=== Environment Example ==="
cat .env.example

echo -e "\n=== Main Entry Point ==="
cat nb-dt-import.py

echo -e "\n=== Settings Configuration ==="
cat settings.py

echo -e "\n=== Checking for subprocess/os.system calls (command injection risk) ==="
rg -n "subprocess|os\.system|os\.popen|eval|exec" --type py -C 2

echo -e "\n=== Checking for unsafe file operations ==="
rg -n "open\(|pickle\.load|yaml\.load\(" --type py -C 1 | head -40

Length of output: 7631


🏁 Script executed:

#!/bin/bash

echo "=== Checking repo.py and netbox_api.py for detailed security review ==="
cat repo.py

echo -e "\n=== Checking netbox_api.py for file handling and error management ==="
rg -n -A 5 -B 2 "def upload_images" netbox_api.py

echo -e "\n=== Check for SSL verification usage ==="
rg -n "verify=" --type py -C 2

echo -e "\n=== Check git operations in repo.py ==="
rg -n "git\.|Repo\(" --type py -C 3

echo -e "\n=== Check Makefile for any concerns ==="
cat Makefile

echo -e "\n=== Check if there are any TODO/FIXME comments ==="
rg -n "TODO|FIXME|XXX|HACK" --type py

Length of output: 8559


@marcinpsk I've completed a comprehensive security review of the repository. Here are my findings:

🔴 High Priority Security Issues

1. Resource Leak in Image Upload (netbox_api.py:742)

Issue: Files are opened but never closed, leading to potential resource exhaustion.

files = {i: (os.path.basename(f), open(f, "rb")) for i, f in images.items()}
response = requests.patch(url, headers=headers, files=files, verify=(not self.ignore_ssl))

Recommendation: Use context managers to ensure files are properly closed:

def upload_images(self, baseurl, token, images, device_type):
    """Upload front_image and/or rear_image for the given device type"""
    url = f"{baseurl}/api/dcim/device-types/{device_type}/"
    headers = {"Authorization": f"Token {token}"}
    
    # Open files with proper cleanup
    file_handles = {}
    try:
        file_handles = {i: (os.path.basename(f), open(f, "rb")) for i, f in images.items()}
        response = requests.patch(url, headers=headers, files=file_handles, verify=(not self.ignore_ssl))
        self.handle.log(f"Images {images} updated at {url}: {response}")
    finally:
        # Ensure all files are closed
        for _, (_, fh) in file_handles.items():
            fh.close()

2. Sensitive Data Exposure in Error Messages (nb-dt-import.py:56)

Issue: Error messages include the entire environment variable dictionary, potentially exposing secrets.

f'Environment variable "{var}" is not set.\n\nMANDATORY_ENV_VARS: {str(settings.MANDATORY_ENV_VARS)}.\n\nCURRENT_ENV_VARS: {str(os.environ)}'

Recommendation: Remove environment variable exposure:

f'Environment variable "{var}" is not set.\n\nMANDATORY_ENV_VARS: {str(settings.MANDATORY_ENV_VARS)}'

🟡 Medium Priority Security Issues

3. Dockerfile Running as Root

Issue: The container runs as root user (default), which violates the principle of least privilege.

Recommendation: Add a non-root user to the Dockerfile:

# After COPY *.py ./
RUN useradd -m -u 1000 appuser && chown -R appuser:appuser /app
USER appuser

4. SSL Verification Can Be Disabled

Issue: The IGNORE_SSL_ERRORS setting allows disabling SSL certificate verification, which can expose traffic to man-in-the-middle attacks.

Recommendation:

  • Add a warning in the code when SSL verification is disabled
  • Consider removing this option for production environments
  • Update .env.example to warn about the security implications

5. Unvalidated Git Repository URL

Issue: Repository URL from environment variables lacks validation, potentially allowing clone from untrusted sources.

Recommendation: Add URL validation in repo.py:

def validate_git_url(self, url):
    """Validate git URL to prevent malicious repositories"""
    import re
    # Allow only https URLs or known safe patterns
    if not re.match(r'^https://github\.com/[\w-]+/[\w-]+\.git$', url):
        self.handle.exception(
            "InvalidGitURL",
            url,
            "Git URL must be a valid HTTPS GitHub URL"
        )

🔵 Low Priority Issues & Best Practices

6. GitHub Actions Workflow Permissions

Current: Workflows use default permissions.

Recommendation: Add explicit minimal permissions to workflow files:

permissions:
  contents: read
  packages: write  # Only for docker.yml

7. Missing Dependency Vulnerability Scanning

Recommendation: Add Dependabot configuration:

# .github/dependabot.yml
version: 2
updates:
  - package-ecosystem: "pip"
    directory: "/"
    schedule:
      interval: "weekly"
  - package-ecosystem: "github-actions"
    directory: "/"
    schedule:
      interval: "weekly"

⚠️ Inconsistency Found

Workflow File Mismatch

The PR diff shows tests.yml with:

- name: Run tests
  run: uv run pytest -vv -s

But the actual file content shows:

- name: Run tests
  run: uv run pytest 

Action Required: The file content doesn't match the PR changes. Please verify the correct state of the workflow file.

✅ Security Positives

  1. ✓ No hardcoded secrets found
  2. ✓ Proper use of environment variables for credentials
  3. ✓ Using yaml.safe_load() instead of unsafe yaml.load()
  4. ✓ Modern dependency management with uv
  5. ✓ Test mocks use dummy tokens
  6. ✓ Dockerfile follows multi-stage build best practices
  7. ✓ No command injection vulnerabilities found (subprocess calls are safe)
  8. ✓ Using official Python slim images

Summary

Critical: 2 issues (resource leak, data exposure)
Medium: 3 issues (root user, SSL bypass, unvalidated URL)
Low: 2 recommendations (permissions, dependency scanning)
Inconsistency: 1 (workflow file mismatch)

Would you like me to create GitHub issues to track the remediation of these security findings?

🐰

@marcinpsk
Copy link
Owner Author

@coderabbitai generate docstrings

@coderabbitai
Copy link

coderabbitai bot commented Dec 14, 2025

✅ Actions performed

Initiated docstring generation; will generate only if new commits exist.

@coderabbitai
Copy link

coderabbitai bot commented Dec 14, 2025

Note

Docstrings generation - SUCCESS
Generated docstrings for this pull request at #4

@marcinpsk marcinpsk changed the title chore: update Docker image reference and test output in CI workflows chore: update CI workflows, add pytest-timeout, and switch to ThreadPoolExecutor Dec 14, 2025
Docstrings generation was requested by @marcinpsk.

* #1 (comment)

The following files were modified:

* `log_handler.py`
* `nb-dt-import.py`
* `netbox_api.py`
* `repo.py`
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
nb-dt-import.py (1)

51-57: Your improved missing-env message only shows up with --verbose.
Because LogHandler.exception() exits with exception_dict["EnvironmentError"], the Line 56 string is only printed when args.verbose is true. If you want the MANDATORY_ENV_VARS hint always, pass it as the main exception message (or update LogHandler.exception to incorporate stack_trace for EnvironmentError).

One minimal option (no env dump, always visible):

-            handle.exception(
-                "EnvironmentError",
-                var,
-                f'Environment variable "{var}" is not set.\n\nMANDATORY_ENV_VARS: {str(settings.MANDATORY_ENV_VARS)}\n',
-            )
+            handle.exception(
+                "Exception",
+                f'Environment variable "{var}" is not set. MANDATORY_ENV_VARS: {settings.MANDATORY_ENV_VARS}',
+            )
repo.py (1)

134-145: Ensure progress and results stay aligned (avoid silent truncation via zip).

zip(items_iterator, results) will stop at the shorter iterator. Since results is always based on files but items_iterator may be a different iterator/wrapper, any mismatch can drop parsed items without an error.

If progress is meant to be a wrapper over the same files list, consider explicitly asserting lengths (when possible) or driving both sides from the same iterable:

-            items_iterator = progress if progress is not None else files
-            results = executor.map(parse_single_file, files)
+            items_iterator = progress if progress is not None else files
+            results = executor.map(parse_single_file, list(files))
 
             for _, data in zip(items_iterator, results):

Or, if progress yields the file paths themselves, feed that iterator into executor.map (but then don’t also consume it via zip; iterate once).

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9c72fa4 and 92fea33.

📒 Files selected for processing (9)
  • .github/dependabot.yml (1 hunks)
  • .github/workflows/docker.yml (3 hunks)
  • .github/workflows/tests.yml (1 hunks)
  • Dockerfile (1 hunks)
  • env.example (1 hunks)
  • log_handler.py (1 hunks)
  • nb-dt-import.py (1 hunks)
  • netbox_api.py (1 hunks)
  • repo.py (3 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
netbox_api.py (1)
log_handler.py (1)
  • log (35-36)
repo.py (1)
log_handler.py (1)
  • exception (12-26)
🔇 Additional comments (3)
.github/dependabot.yml (1)

4-11: Dependabot config update looks good (pip + GitHub Actions).
Weekly checks for both ecosystems is a sensible baseline.

log_handler.py (1)

13-20: InvalidGitURL mapping is clear and actionable.
Nice improvement in error specificity while keeping behavior consistent.

.github/workflows/docker.yml (1)

4-6: Explicit minimal permissions: good hardening.
This is the right direction for GITHUB_TOKEN least-privilege.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
nb-dt-import.py (1)

1-1: Critical: Black formatting check failed.

The pipeline reports that this file would be reformatted by Black. Run black nb-dt-import.py locally and commit the formatted version to fix this blocking issue.

log_handler.py (1)

1-1: Critical: Black formatting check failed.

The pipeline reports that this file would be reformatted by Black. Run black log_handler.py locally and commit the formatted version to fix this blocking issue.

netbox_api.py (1)

1-1: Critical: Black formatting check failed.

The pipeline reports that this file would be reformatted by Black. Run black netbox_api.py locally and commit the formatted version to fix this blocking issue.

♻️ Duplicate comments (3)
netbox_api.py (1)

744-754: Critical: Dict-comprehension still leaks file handles on partial failure.

The past review comment is still valid. If the dict-comprehension at line 747 fails partway through (e.g., second file doesn't exist), the successfully opened file handles are leaked because file_handles is never fully assigned and the finally block won't see them.

Apply the fix from the previous review:

-        # Open files with proper cleanup to avoid resource leaks
-        file_handles = {}
-        try:
-            file_handles = {i: (os.path.basename(f), open(f, "rb")) for i, f in images.items()}
-            response = requests.patch(url, headers=headers, files=file_handles, verify=(not self.ignore_ssl))
-            self.handle.log(f"Images {images} updated at {url}: {response}")
-            self.counter["images"] += len(images)
-        finally:
-            # Ensure all file handles are closed
-            for _, (_, fh) in file_handles.items():
-                fh.close()
+        # Open files with proper cleanup to avoid resource leaks
+        file_handles = {}
+        try:
+            for field, path in images.items():
+                file_handles[field] = (os.path.basename(path), open(path, "rb"))
+            response = requests.patch(url, headers=headers, files=file_handles, verify=(not self.ignore_ssl))
+            self.handle.log(f"Images {images} updated at {url}: {response}")
+            self.counter["images"] += len(images)
+        finally:
+            for _, (_, fh) in file_handles.items():
+                try:
+                    fh.close()
+                except Exception:
+                    pass
repo.py (2)

80-84: Major: URL validation blocks pull-only usage.

The past review comment is still valid. URL validation runs before checking if repo_path exists, which means an invalid or empty URL will exit even if you could proceed with pull_repo() on an existing local clone.

Move validation inside the clone path:

-        # Validate URL before cloning
-        is_valid, error_msg = validate_git_url(self.url)
-        if not is_valid:
-            self.handle.exception("InvalidGitURL", self.url, error_msg)
-
         if os.path.isdir(self.repo_path):
             self.pull_repo()
         else:
+            # Validate URL before cloning
+            is_valid, error_msg = validate_git_url(self.url)
+            if not is_valid:
+                self.handle.exception("InvalidGitURL", self.url, error_msg)
             self.clone_repo()

10-28: Major: URL validation is too weak and rejects valid URLs.

The past review comment is still valid. The current regex-based validation:

  • Rejects valid URLs with explicit ports (e.g., https://github.com:443/org/repo.git)
  • Allows arbitrary internal hosts (potential SSRF risk)
  • Doesn't handle whitespace or empty strings robustly

Use urllib.parse for robust validation:

+from urllib.parse import urlparse
+
 def validate_git_url(url):
     """
     Validate that a git remote URL uses HTTPS or SSH and reject unsafe or local schemes.
     
     Returns:
         tuple: (bool, str or None) — (True, None) if the URL is allowed; (False, error_message) otherwise.
     """
-    if not url:
+    if not url or not str(url).strip():
         return False, "Empty URL"
+    url = str(url).strip()
 
     # Allow HTTPS
-    if re.match(r"^https://[\w.-]+/", url):
-        return True, None
+    if url.startswith("https://"):
+        parsed = urlparse(url)
+        if parsed.scheme == "https" and parsed.hostname:
+            return True, None
+        return False, "Invalid HTTPS URL"
 
     # Allow SSH patterns
-    if re.match(r"^git@[\w.-]+:", url) or re.match(r"^ssh://", url):
-        return True, None
+    if url.startswith("git@"):
+        # scp-like syntax: git@host:org/repo.git
+        return (":" in url.split("@", 1)[-1]), None if (":" in url.split("@", 1)[-1]) else (False, "Invalid git@ URL")
+    if url.startswith("ssh://"):
+        parsed = urlparse(url)
+        if parsed.scheme == "ssh" and parsed.hostname:
+            return True, None
+        return False, "Invalid SSH URL"
 
-    return False, f"URL must use HTTPS or SSH protocol, got: {url}"
+    return False, "URL must use HTTPS or SSH protocol"

Consider also adding a hostname allowlist (e.g., github.com) if appropriate for your threat model.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 92fea33 and aead89f.

📒 Files selected for processing (4)
  • log_handler.py (2 hunks)
  • nb-dt-import.py (3 hunks)
  • netbox_api.py (1 hunks)
  • repo.py (5 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
repo.py (1)
log_handler.py (1)
  • exception (12-37)
netbox_api.py (1)
log_handler.py (1)
  • log (46-47)
🪛 GitHub Actions: tests
log_handler.py

[error] 1-1: Black formatting check failed. File would be reformatted by Black: /home/runner/work/Device-Type-Library-Import/Device-Type-Library-Import/log_handler.py

repo.py

[error] 1-1: Black formatting check failed. File would be reformatted by Black: /home/runner/work/Device-Type-Library-Import/Device-Type-Library-Import/repo.py

netbox_api.py

[error] 1-1: Black formatting check failed. File would be reformatted by Black: /home/runner/work/Device-Type-Library-Import/Device-Type-Library-Import/netbox_api.py

nb-dt-import.py

[error] 1-1: Black formatting check failed. File would be reformatted by Black: /home/runner/work/Device-Type-Library-Import/Device-Type-Library-Import/nb-dt-import.py

🔇 Additional comments (4)
nb-dt-import.py (1)

58-62: Good fix: removed sensitive environment variable dump.

This change addresses the critical security issue flagged in the PR objectives where CURRENT_ENV_VARS (containing all environment variables) was logged in error messages. The new message only includes the missing variable name and the list of mandatory variables.

log_handler.py (1)

12-37: LGTM: Well-documented exception handler with new InvalidGitURL support.

The docstring clearly describes parameters and behavior. The new InvalidGitURL exception mapping properly integrates with the URL validation added in repo.py, providing a clear error message with optional context from the validator.

repo.py (2)

31-54: LGTM: Well-documented helper function for YAML parsing.

The extraction of parse_single_file improves testability and the docstring clearly describes behavior, parameters, and return values.


160-181: LGTM: ThreadPoolExecutor aligns with pytest timeout configuration.

The switch to ThreadPoolExecutor is appropriate for the mixed I/O and YAML parsing workload, avoiding GIL-related complications with the thread-based timeout configuration (timeout_method = "thread" in pyproject.toml). The docstring accurately describes the parallel execution behavior.

marcinpsk and others added 2 commits December 14, 2025 15:25
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
@marcinpsk
Copy link
Owner Author

@coderabbitai generate docstrings

@coderabbitai
Copy link

coderabbitai bot commented Dec 14, 2025

✅ Actions performed

Initiated docstring generation; will generate only if new commits exist.

Docstrings generation was requested by @marcinpsk.

* #1 (comment)

The following files were modified:

* `log_handler.py`
* `netbox_api.py`
* `repo.py`
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ce8dcb7 and 4d3cbf0.

📒 Files selected for processing (6)
  • Dockerfile (1 hunks)
  • env.example (1 hunks)
  • log_handler.py (1 hunks)
  • nb-dt-import.py (2 hunks)
  • netbox_api.py (4 hunks)
  • repo.py (5 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
netbox_api.py (1)
log_handler.py (1)
  • log (52-53)
repo.py (1)
log_handler.py (1)
  • exception (18-43)
🔇 Additional comments (17)
Dockerfile (1)

30-31: ✅ Solid non-root hardening implementation.

The non-root user creation and ownership transfer follow proper sequence: RUN commands execute as root (creating the user and changing /app ownership), then USER appuser switches subsequent operations to run as a non-privileged user (UID 1000). This addresses the security concern effectively. The line endings are correct (LF), so no build issues will occur.

env.example (1)

1-12: LGTM! Well-documented configuration template.

The environment variables are clearly documented with helpful comments:

  • NetBox URL format guidance (no trailing slash)
  • Required API token permissions explicitly listed
  • Sensible defaults for repository and branch
  • All values are appropriate placeholders/examples

This provides good guidance for users setting up the integration.

netbox_api.py (4)

572-577: LGTM! Improved observability for skipped front ports.

The added logging provides clear visibility when front ports are skipped due to invalid rear port references, including context information. This helps with debugging device type definitions.


695-704: LGTM! Clear and comprehensive docstring.

The docstring accurately describes the function's behavior, parameters, and the rear port resolution logic. The documentation of how unresolved rear ports are handled is particularly helpful.


734-739: LGTM! Consistent logging for module front ports.

This mirrors the logging improvements for device-type front ports, maintaining consistency across the codebase.


752-787: LGTM! Resource leak fixed with robust file handling.

The refactored implementation correctly addresses the critical resource leak identified in previous reviews. Files are now opened one-by-one in a loop (lines 771-779), with proper cleanup of already-opened files if any open fails (lines 775-778), and a finally block ensures all handles are closed (lines 784-787). The comprehensive docstring clearly documents the function's behavior and parameters.

log_handler.py (3)

10-15: LGTM! Clear initialization docstring.

The docstring concisely documents the parameter and its purpose.


18-29: LGTM! Comprehensive exception method documentation.

The docstring clearly describes the method's behavior, all parameters including the optional stack_trace, and the raised SystemExit exception. This significantly improves the maintainability of the error handling logic.


35-35: LGTM! Enhanced error message for invalid Git URLs.

The updated message now includes the URL and either the provided stack_trace or a helpful default message. This aligns well with the new validate_git_url function in repo.py and provides better debugging context.

repo.py (6)

10-50: LGTM! Robust URL validation implementation.

The validate_git_url function properly validates Git URLs using urlparse for HTTPS and handles SSH variants (scp-like git@host: and ssh:// schemes). The implementation:

  • Strips whitespace from input
  • Validates HTTPS URLs by checking scheme and hostname
  • Validates SSH scp-like syntax by checking for colon after the host
  • Validates SSH URLs using urlparse
  • Returns clear error messages for each failure case

The commented allowlist on lines 29-31 provides a good extension point for future security hardening if needed.


52-75: LGTM! Clean YAML parsing with proper error handling.

The parse_single_file function correctly loads YAML files, normalizes the manufacturer field to a slug dictionary for consistent lookups, and handles both YAML parsing errors and general exceptions. The docstring clearly documents the return types and behavior.


80-103: LGTM! Well-documented initialization.

The docstrings for __new__ and __init__ clearly describe the initialization flow. Importantly, the implementation now validates the URL only when cloning a new repository (lines 117-119), not when pulling from an existing local repository (lines 113-114). This correctly addresses the past review concern about blocking pull-only usage.


122-128: LGTM! Concise docstring.

The docstring clearly documents the method's return value.


172-188: LGTM! Comprehensive docstring for get_devices.

The docstring thoroughly documents the function's behavior, parameters, return structure, and the "testing" folder exclusion rule. This significantly improves code maintainability.


200-215: LGTM! parse_files improvements.

The comprehensive docstring documents the parameters, filtering behavior, and return semantics. Line 214's comment correctly states "ThreadPoolExecutor" matching the implementation on line 215, which addresses the past review concern. The choice of ThreadPoolExecutor is appropriate for this mixed I/O and CPU workload (file reading + YAML parsing).

nb-dt-import.py (2)

23-29: LGTM! Clear and concise docstring.

The docstring provides a brief summary followed by a clear list of the main orchestration steps. This addresses the past review suggestion to condense the documentation while maintaining clarity.


63-63: LGTM! Improved error message security.

The updated error message no longer exposes the full environment variables dump (CURRENT_ENV_VARS from os.environ), addressing the security concern noted in the PR objectives. The message now provides only the necessary information: the missing variable name and the list of mandatory environment variables.

@marcinpsk
Copy link
Owner Author

@coderabbitai generate docstrings

@coderabbitai
Copy link

coderabbitai bot commented Dec 14, 2025

✅ Actions performed

Initiated docstring generation; will generate only if new commits exist.

Docstrings generation was requested by @marcinpsk.

* #1 (comment)

The following files were modified:

* `log_handler.py`
* `netbox_api.py`
* `repo.py`
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (9)
log_handler.py (3)

24-50: Harden exception() against unknown exception_type and avoid leaking “stack_trace” unintentionally.
exception_dict[exception_type] can raise KeyError, and "InvalidGitURL" currently inlines stack_trace into the user-facing message even when not verbose. Consider a safe fallback and treat stack_trace as “details” unless you truly want to expose it.

-        exception_dict = {
+        exception_dict = {
             "EnvironmentError": f'Environment variable "{exception}" is not set.',
             "SSLError": f"SSL verification failed. IGNORE_SSL_ERRORS is {exception}. Set IGNORE_SSL_ERRORS to True if you want to ignore this error. EXITING.",
             "GitCommandError": f'The repo "{exception}" is not a valid git repo.',
             "GitInvalidRepositoryError": f'The repo "{exception}" is not a valid git repo.',
-            "InvalidGitURL": f'Invalid Git URL: {exception}. {stack_trace or "URL must use HTTPS or SSH protocol."}',
+            "InvalidGitURL": f'Invalid Git URL: {exception}. {stack_trace or "URL must use HTTPS or SSH protocol."}',
             "Exception": f'An unknown error occurred: "{exception}"',
         }
 
-        if self.args.verbose and stack_trace:
+        if self.args.verbose and stack_trace:
             print(stack_trace)
 
         # Raise SystemExit with the message, which will print to stderr and exit code 1
-        system_exit(exception_dict[exception_type])
+        system_exit(exception_dict.get(exception_type, exception_dict["Exception"]))

61-77: Avoid mutable default args for created_ports.
Even if you don’t mutate today, this is a footgun.

-    def log_device_ports_created(self, created_ports: list = [], port_type: str = "port"):
+    def log_device_ports_created(self, created_ports: list | None = None, port_type: str = "port"):
+        created_ports = created_ports or []
         for port in created_ports:
             self.verbose_log(
                 f"{port_type} Template Created: {port.name} - "
                 + f'{port.type if hasattr(port, "type") else ""} - {port.device_type.id} - '
                 + f"{port.id}"
             )
         return len(created_ports)
 
-    def log_module_ports_created(self, created_ports: list = [], port_type: str = "port"):
+    def log_module_ports_created(self, created_ports: list | None = None, port_type: str = "port"):
+        created_ports = created_ports or []
         for port in created_ports:
             self.verbose_log(
                 f"{port_type} Template Created: {port.name} - "
                 + f'{port.type if hasattr(port, "type") else ""} - {port.module_type.id} - '
                 + f"{port.id}"
             )
         return len(created_ports)

1-77: CI blocker: Black would reformat this file.
Please run Black on log_handler.py and commit the result to unblock the workflow.

repo.py (4)

52-76: Fix parse_single_file() docstring/typing and handle non-mapping YAML defensively.
Right now the docstring implies two return values; the function actually returns a single dict | str. Also consider encoding="utf-8" and guarding when YAML returns None/non-dict.

 def parse_single_file(file):
     """
@@
-    Returns:
-        dict: Parsed mapping with `manufacturer` replaced by `{"slug": "<slugified-name>"}` and `src` set to the file path.
-        str: Error string beginning with "Error:" describing YAML parsing or other failure.
+    Returns:
+        dict | str: Parsed mapping (dict) with `manufacturer` replaced by `{"slug": "<slug>"}` and `src` set,
+        or an error string beginning with "Error:".
     """
-    with open(file, "r") as stream:
+    with open(file, "r", encoding="utf-8") as stream:
         try:
             data = yaml.safe_load(stream)
+            if not isinstance(data, dict):
+                return "Error: YAML root must be a mapping"
             manufacturer = data["manufacturer"]
@@
             return data

88-116: DTLRepo.__init__ docstring doesn’t match the real exception handler signature.
It says exception(name, context, message) but LogHandler.exception() is (exception_type, exception, stack_trace=None); please align the wording to avoid misuse.


206-241: parse_files() slug filtering is fragile (possible KeyError) and unnecessarily complex.
If a YAML lacks "slug", data["slug"] will crash when slugs is provided. Also replace the list-comp boolean pattern with any(...).

-                if slugs and True not in [True if s.casefold() in data["slug"].casefold() else False for s in slugs]:
+                if slugs:
+                    item_slug = str(data.get("slug", ""))
+                    if not any(s.casefold() in item_slug.casefold() for s in slugs):
                     self.handle.verbose_log(f"Skipping {data['model']}")
                     continue

178-205: get_devices() vendor filtering breaks case-insensitive matching and path building can silently omit files.

  1. Vendor filtering logic is inverted: Line 199 compares vendor.casefold() against the raw vendors list. Since vendors contains original-case strings (e.g., ["NetBox"]), the condition "netbox" in ["NetBox"] always fails. This silently skips vendors that don't match the exact case of the input list, contradicting the docstring's "case-insensitive" claim.

  2. Path concatenation breaks without trailing separator: Line 203 uses base_path + folder + f"/*.{extension}". If base_path="./devices" (no trailing slash), the result becomes "./devicesvendor/*.yaml" instead of "./devices/vendor/*.yaml", causing glob() to fail silently and miss all files.

Apply the suggested fixes:

  • Convert vendors to a casefolded set for correct comparison
  • Use os.path.join() for robust path construction
netbox_api.py (2)

1-815: CI blocker: Black would reformat this file.
Please run Black on netbox_api.py and commit the result.


544-598: Rear-port resolution fails in the non-cache path due to inconsistent dict keying.
When cache is unavailable, existing_rp = {str(item): item ...} creates keys that don't match port["rear_port"] (a name string). Use item.name consistently in both paths. Also guard access to "rear_port" and "name" keys in the except block to prevent cascading errors.

             else:
                 rp_endpoint = self.netbox.dcim.rear_port_templates
                 rp_kwargs = self._get_filter_kwargs(pid, "device")
-                existing_rp = {str(item): item for item in rp_endpoint.filter(**rp_kwargs)}
+                existing_rp = {item.name: item for item in rp_endpoint.filter(**rp_kwargs)}
@@
             for port in items:
                 try:
-                    rear_port = existing_rp[port["rear_port"]]
+                    rear_port_name = port.get("rear_port")
+                    rear_port = existing_rp[rear_port_name]
                     port["rear_port"] = rear_port.id
                 except KeyError:
                     available = list(existing_rp.keys()) if existing_rp else []
                     ctx = f" (Context: {context})" if context else ""
                     self.handle.log(
-                        f'Could not find Rear Port "{port["rear_port"]}" for Front Port "{port["name"]}". '
+                        f'Could not find Rear Port "{port.get("rear_port")}" for Front Port "{port.get("name")}". '
                         f"Available: {available}{ctx}"
                     )
                     ports_to_remove.append(port)

This pattern appears in both create_front_ports and create_module_front_ports.

♻️ Duplicate comments (1)
repo.py (1)

1-241: CI blocker (repeat): Black would reformat this file.
Same as earlier feedback—please re-run Black on repo.py and commit the result.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 4d3cbf0 and d8603f6.

📒 Files selected for processing (3)
  • log_handler.py (2 hunks)
  • netbox_api.py (6 hunks)
  • repo.py (7 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
netbox_api.py (1)
log_handler.py (1)
  • log (58-59)
repo.py (1)
log_handler.py (1)
  • exception (24-49)
🪛 GitHub Actions: tests
netbox_api.py

[error] 1-1: Black formatting would reformat this file.

log_handler.py

[error] 1-1: Black formatting would reformat this file.

repo.py

[error] 1-1: Black formatting would reformat this file.

🔇 Additional comments (2)
repo.py (1)

10-49: validate_git_url() looks reasonable; add targeted tests for the accepted/rejected forms.
Especially cover whitespace, git@host:org/repo.git, ssh://user@host/path, and invalid git@host/org.

netbox_api.py (1)

693-779: The device and module front-port linking implementations already use identical patterns (direct key access with try/except KeyError). No changes needed; the code is consistent as-is.

Likely an incorrect or invalid review comment.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
netbox_api.py (1)

781-819: Avoid logging full filesystem paths for uploaded images

The refactor here largely addresses the earlier concerns (file-handle cleanup, timeout=60, and response.raise_for_status() before incrementing the counter). However, the log line still includes the raw images mapping:

self.handle.log(f"Images {images} updated at {url}: {response.status_code}")

This exposes full local filesystem paths in logs, which was previously flagged as a security/privacy issue (leaking internal layout, usernames, etc.).

You already have basenames available; consider logging only those and basing the counter on successfully opened handles:

-            self.handle.log(f"Images {images} updated at {url}: {response.status_code}")
-            self.counter["images"] += len(images)
+            safe_image_names = {field: os.path.basename(path) for field, path in images.items()}
+            self.handle.log(f"Images {safe_image_names} updated at {url}: {response.status_code}")
+            self.counter["images"] += len(file_handles)

This keeps operational visibility while avoiding path disclosure and ties the counter to what was actually attached to the request.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 0bf196c and 287bfb1.

📒 Files selected for processing (2)
  • log_handler.py (1 hunks)
  • netbox_api.py (6 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
netbox_api.py (1)
log_handler.py (1)
  • log (56-57)
🔇 Additional comments (6)
log_handler.py (3)

6-11: Well-documented class.

The class docstring clearly describes the purpose and key features of the LogHandler.


23-33: Comprehensive method documentation.

The docstring thoroughly documents parameters, behavior, and side effects.


43-44: Good addition for debugging support.

The conditional printing of stack traces when verbose mode is enabled is helpful for debugging, though it depends on resolving the parameter misuse on line 39.

netbox_api.py (3)

543-598: Device front-port docstring and skipped-port logging look correct

The new docstring accurately describes the rear-port resolution / pruning behavior, and the additional summary log for skipped ports (lines 583–588) will make YAML misconfigurations much easier to debug without changing behavior.


694-712: Module rear-port docstring matches existing behavior

The added docstring clearly documents that this helper only creates missing rear-port templates for a module type and delegates to _create_generic, which aligns with how the function already behaved. This improves readability without altering logic.


714-779: Module front-port docs and skipped-port logging are consistent with implementation

The outer and inner docstrings correctly describe resolving rear_port names to IDs, pruning unresolved module front ports, and logging both individual misses and a final summary. The new summary log for skipped module front ports (lines 763–769) mirrors the device-level behavior and should aid troubleshooting.

marcinpsk and others added 4 commits December 15, 2025 12:22
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant