Skip to content

[management] Read reverse-proxy service and target columns in Postgres path - #6886

Merged
lixmal merged 4 commits into
mainfrom
fix-pgx-service-column-parity
Jul 28, 2026
Merged

[management] Read reverse-proxy service and target columns in Postgres path#6886
lixmal merged 4 commits into
mainfrom
fix-pgx-service-column-parity

Conversation

@lixmal

@lixmal lixmal commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Describe your changes

The Postgres read path in getServices (sql_store.go) hand-writes the column list and row scan, and had drifted from the gorm model. Several service and target columns were absent from the SELECT and scan, so on Postgres those fields came back zero-valued while SQLite and MySQL loaded them correctly. This adds the missing columns and a test that prevents the lists from drifting again.

  • Load service access restrictions (allowed/blocked CIDRs and countries, CrowdSec mode) on the Postgres path
  • Load the service meta_last_renewed_at timestamp used for ephemeral service expiry
  • Load target proxy_protocol and the full embedded target options (TLS skip, timeouts, path rewrite, custom headers, direct upstream, capture settings, middlewares, agent-network flags)
  • Move the service/target column lists into shared constants and split the loader into smaller helpers
  • Add a test that fails when a gorm column is missing from the Postgres column list, plus a round-trip test that fails on Postgres without the fix

Issue ticket number and link

Stack

Checklist

  • Is it a bug fix
  • Is a typo/documentation fix
  • Is a feature enhancement
  • It is a refactor
  • Created tests that fail without the change (if possible)
  • This change does not modify the public API, gRPC protocols, functionality behavior, CLI / service flags, or introduce a new feature — OR I have discussed it with the NetBird team beforehand (link the issue / Slack thread in the description). See CONTRIBUTING.md.

By submitting this pull request, you confirm that you have read and agree to the terms of the Contributor License Agreement.

Documentation

Select exactly one:

  • I added/updated documentation for this change
  • Documentation is not needed for this change (explain why)

Internal storage-layer parity fix with no user-facing or API change.

Docs PR URL (required if "docs added" is checked)

Paste the PR link from https://github.com/netbirdio/docs here:

https://github.com/netbirdio/docs/pull/__

Summary by CodeRabbit

  • Bug Fixes
    • Improved reliability of service loading so metadata (including renewal timestamps), restrictions, and related fields are consistently preserved.
    • Enhanced target option loading for timeouts, TLS behavior, path rewrite mode, custom headers/middleware, capture limits/content types, agent-network settings, and access-log preferences.
    • More robust JSON handling with better error clarity and safer validation of numeric fields.
  • Tests
    • Added round-trip test coverage for service/target options across supported database engines.
    • Added a Postgres pgx vs ORM column parity check to prevent accidentally omitted columns.

@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 48fa4a24-d6ac-423c-a889-ab8d26e639d5

📥 Commits

Reviewing files that changed from the base of the PR and between 828df02 and cd983c5.

📒 Files selected for processing (1)
  • management/server/store/sql_store.go

📝 Walkthrough

Walkthrough

The SQL store now reloads service restrictions, renewal metadata, and comprehensive target options, including scalar and JSON fields. Explicit PGX column lists are checked against GORM model columns, and roundtrip tests verify restored values across supported database engines.

Changes

SQL service loading

Layer / File(s) Summary
Service metadata loading
management/server/store/sql_store.go
Explicit service columns and scanService now load restrictions, renewal metadata, and JSON fields with guarded decoding.
Target option mapping and validation
management/server/store/sql_store.go, management/server/store/sql_store_service_test.go
Targets are loaded in a separate query, mapped into service options, attached by service ID, and verified through persistence roundtrip assertions.
PGX column parity checks
management/server/store/sql_store_pgx_parity_test.go
A test compares PGX-selected columns with GORM-derived service and target column names.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant RoundtripTest
  participant GetAccount
  participant getServices
  participant SQLDatabase
  RoundtripTest->>GetAccount: request persisted account
  GetAccount->>getServices: load services and targets
  getServices->>SQLDatabase: query service metadata
  SQLDatabase-->>getServices: service rows
  getServices->>SQLDatabase: query target option columns
  SQLDatabase-->>getServices: target rows
  getServices-->>GetAccount: hydrated account
  GetAccount-->>RoundtripTest: restored restrictions and target options
Loading

Possibly related PRs

  • netbirdio/netbird#6842: Both changes modify SQL service and target hydration and column selection in sql_store.go.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately summarizes the Postgres reverse-proxy service and target column read-path fix.
Description check ✅ Passed The description follows the template, covers the change, checklist, and documentation choice, and is mostly complete despite a blank issue section.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix-pgx-service-column-parity

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
management/server/store/sql_store_service_test.go (1)

91-104: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add Middlewares to the roundtrip coverage.

Options.Middlewares is a newly-mapped JSON field (guarded unmarshal added in sql_store.go, mirroring CustomHeaders/CaptureContentTypes) but isn't set on the target here nor asserted in the verification block, so its round trip is unverified.

♻️ Proposed addition
 					Options: rpservice.TargetOptions{
 						SkipTLSVerify:           true,
 						RequestTimeout:          30 * time.Second,
 						SessionIdleTimeout:      5 * time.Minute,
 						PathRewrite:             rpservice.PathRewritePreserve,
 						CustomHeaders:           map[string]string{"X-Foo": "bar"},
 						DirectUpstream:          true,
+						Middlewares:             []string{"rate-limit"},
 						CaptureMaxRequestBytes:  1024,
 						CaptureMaxResponseBytes: 2048,
 						CaptureContentTypes:     []string{"application/json"},
 						AgentNetwork:            true,
 						DisableAccessLog:        true,
 					},
 		assert.True(t, tg.Options.DirectUpstream, "options direct upstream")
+		assert.Equal(t, []string{"rate-limit"}, tg.Options.Middlewares, "options middlewares")
 		assert.Equal(t, int64(1024), tg.Options.CaptureMaxRequestBytes, "options capture max request bytes")

Adjust the field type/value above to match the actual Middlewares field type on TargetOptions.

Also applies to: 120-133

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@management/server/store/sql_store_service_test.go` around lines 91 - 104, Add
Middlewares to the TargetOptions fixture in the round-trip test using a value
matching its actual field type, then update the corresponding verification block
to assert the value survives serialization and deserialization. Apply the same
coverage to the additional target fixture referenced by the comment.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@management/server/store/sql_store_service_test.go`:
- Around line 91-104: Add Middlewares to the TargetOptions fixture in the
round-trip test using a value matching its actual field type, then update the
corresponding verification block to assert the value survives serialization and
deserialization. Apply the same coverage to the additional target fixture
referenced by the comment.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: a600a040-9fb3-4538-93a1-fbd422524b58

📥 Commits

Reviewing files that changed from the base of the PR and between 1e5b0a5 and 9a5691f.

📒 Files selected for processing (2)
  • management/server/store/sql_store.go
  • management/server/store/sql_store_service_test.go

@github-actions

github-actions Bot commented Jul 24, 2026

Copy link
Copy Markdown

Release artifacts

Built for PR head cd983c5 in workflow run #17069.

Artifact Link
All release artifacts Download
Linux packages Download
Windows packages Download
macOS packages Download
UI artifacts Download
UI macOS artifacts Download

GHCR images (amd64)

This comment is updated by the Release workflow. Artifact links expire according to the workflow retention policy.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@management/server/store/sql_store.go`:
- Around line 2405-2406: Update scanService’s listenPort handling to validate
listenPort.Int64 is within the uint16 port range (0–65535) before conversion.
Reject out-of-range persisted values instead of narrowing them, while preserving
assignment for valid values.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 65bb3c14-1f89-47c1-a0a0-8b3ad3e50796

📥 Commits

Reviewing files that changed from the base of the PR and between 9a5691f and 982794d.

📒 Files selected for processing (1)
  • management/server/store/sql_store.go

Comment thread management/server/store/sql_store.go
@sonarqubecloud

Copy link
Copy Markdown

@lixmal
lixmal merged commit 9269b56 into main Jul 28, 2026
48 checks passed
@lixmal
lixmal deleted the fix-pgx-service-column-parity branch July 28, 2026 12:46
@fosskar fosskar Bot mentioned this pull request Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants