feat(proxy): persistent proxy server mode by Sahilb315 · Pull Request #351 · safedep/pmg

Sahilb315 · 2026-06-24T11:03:14Z

Draft. Supersedes the POC in #347 (which will be closed). This is a clean, rebased branch off main — the POC's messy iteration history is dropped.

What this adds

A long-lived persistent proxy server that intercepts package managers via environment variables (no shims/wrappers), targeting CI/CD pipelines.

pmg proxy start [--daemon|-D] [--state PATH] [--port N]
pmg proxy stop  [--fail-on-violation]
pmg proxy env
pmg proxy status

Typical CI flow:

- uses: safedep/pmg@v1
  with:
    server-mode: true
    api-key: ${{ secrets.SAFEDEP_API_KEY }}
    tenant-id: ${{ secrets.SAFEDEP_TENANT_ID }}
- run: npm ci                                # intercepted via HTTP_PROXY, no wrapper
- run: pmg proxy stop --fail-on-violation    # drains, flushes cloud, fails on block
  if: always()

Highlights

Daemonization (--daemon, Unix; Windows returns not-supported) via detached Setsid re-exec.
Configurable state (--state, defaults to <cache-dir>/proxy-state.json) and --port.
Generic env — plain KEY=VALUE; omits cert vars when the PMG CA is already OS-trusted.
Durable cloud sync — the daemon initializes the audit pipeline so existing interceptors log malware-blocked events; stop performs a synchronous cloud flush so events land before an ephemeral runner is destroyed.
Fail-on-block — stop --fail-on-violation exits non-zero on any block and fails closed if the proxy crashed (unverifiable run).
GitHub Action server-mode on the existing composite action.

Structure

internal/proxyserver/ — all lifecycle/analysis/state/cloud logic (state, Run, Daemonize, Stop, EnvVars, GetStatus, analyzer+cache, cloud flush).
cmd/proxy/ — thin cobra wrappers (flag binding + output only), mirroring the cmd/* → internal/* pattern.
internal/flows/cert.go — SetupCACertificate extracted from proxyFlow for reuse.

Deferred (separate work)

pmg setup install --system + iptables enforcement (PMG Setup Install Fails for User when Installed via. Root #317; blocked on shim-recursion fix).
Windows daemonization.
pmg proxy stop --violations [--json] reporting.

Notes for review

Review feedback from feat(proxy): persistent proxy server (POC) #347 (drop --gha, daemonize, --state/--port, opt-in fail, python3 -m pip E2E) is incorporated.
The proxy reuses the existing interceptor/analyzer flow unchanged; it adds a new entrypoint, not new security logic — so no new test/proxye2e/ case is added. Happy to add one if preferred.

🤖 Generated with Claude Code

Move the CA load/generate/merge logic out of proxyFlow into an exported flows.SetupCACertificate so the persistent proxy server can reuse it.

Introduces 'pmg proxy' commands backed by internal/proxyserver: a long-lived MITM proxy that intercepts package managers via env vars (no shims). Supports --daemon (Unix), --state, --port; generic 'env' output that skips cert vars when the CA is OS-trusted; opt-in 'stop --fail-on-violation' (fail-closed on crash) with a synchronous cloud event flush; and the malysis analysis cache.

When server-mode=true the action starts the proxy daemon and injects proxy env vars into the job instead of installing shims.

codecov-commenter · 2026-06-24T11:05:23Z

Codecov Report

❌ Patch coverage is 21.25237% with 415 lines in your changes missing coverage. Please review.
✅ Project coverage is 54.10%. Comparing base (7ee4187) to head (e6085fc).

Files with missing lines	Patch %	Lines
internal/proxyserver/server.go	0.00%	119 Missing ⚠️
cmd/proxy/start.go	15.38%	44 Missing ⚠️
cmd/proxy/env.go	0.00%	36 Missing ⚠️
internal/proxyserver/stop.go	0.00%	31 Missing ⚠️
cmd/proxy/stop.go	38.29%	29 Missing ⚠️
internal/audit/cloud_client.go	0.00%	24 Missing ⚠️
cmd/proxy/status.go	0.00%	20 Missing ⚠️
internal/flows/cert.go	44.44%	14 Missing and 6 partials ⚠️
internal/proxyserver/daemonize_unix.go	0.00%	20 Missing ⚠️
cmd/proxy/proxy.go	0.00%	11 Missing ⚠️
... and 11 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #351      +/-   ##
==========================================
- Coverage   55.21%   54.10%   -1.11%     
==========================================
  Files         190      205      +15     
  Lines       13518    13937     +419     
==========================================
+ Hits         7464     7541      +77     
- Misses       5360     5697     +337     
- Partials      694      699       +5

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

On a fresh CI runner the cache directory does not exist yet; os.OpenFile and os.WriteFile do not create parent dirs, so 'pmg proxy start --daemon' failed with 'no such file or directory'. MkdirAll the parent before writing.

… status npm/pip/yarn/requests trust the MITM CA inconsistently across tools, versions, and configs; many still use bundled CA stores. Always emitting the cert-path env vars is the conservative choice that works regardless, and is harmless for tools that read the OS store (they ignore the vars). Skipping them when a system CA exists would silently break any tool still on a bundled store.

main.go's PersistentPreRun already initializes the audit pipeline for every command (including the daemon's re-exec'd child) and closes it at process exit. Re-initializing in proxyserver.Run created a second auditor and a second cloud-sync WAL connection, orphaning the first. Removing it makes the daemon consistent with the normal proxy flow, which never self-initializes audit.

safedep · 2026-06-24T13:58:53Z

SafeDep Report Summary

No dependency changes detected. Nothing to scan.

View complete scan results →

_{This report is generated by SafeDep Github App}

pmg proxy stop inherits HTTP(S)_PROXY (injected by 'pmg proxy env') pointing at the PMG proxy it just shut down. The cloud sync gRPC client honored those vars and routed api.safedep.io through the dead proxy, failing with 'connection refused' so no events were delivered. Clear the proxy env vars before the sync so PMG's own cloud traffic goes direct.

- configurable bind host via proxy.server.listen_host (default loopback) - proxy commands use ui.ErrorExit instead of returning errors to cobra - rename errcode to ProxyPolicyViolation (covers malware + cooldown) - share cloud sync via audit.DrainToCloud (de-dup with cmd/cloud/sync) - centralize proxy CA bundle path in certmanager - docs: persistent proxy cert trust + bind address

stopExitError set only WithMsg, but ui.ErrorExit renders HumanError, so the framed error showed 'no human-readable message available'. Set both from one string, and emit the framed error before the stdout summary so the blocked count is stated once.

The stop process inherits HTTP_PROXY (from 'pmg proxy env'), so its cloud client routed api.safedep.io through the already-stopped proxy and failed with connection refused. Move the flush into the daemon's shutdown, which has no proxy env (it started before env injection) and dials SafeDep directly. - daemon flushes on shutdown via audit.DrainToCloud and records the result in the state file; stop surfaces it (on both success and fail-on-violation paths) since the daemon's own logs aren't visible to stop - coordinate stop's wait with the daemon shutdown budget; on timeout, error out without reading stale state or deleting the file (fail-closed) - persist blocked count before the flush so the gate stays correct if the flush hangs or the daemon is killed mid-flush - remove now-redundant cloud_flush.go

- daemon runs a periodic cloud-sync ticker so the shutdown flush stays small; the run total is reported by stop, and shutdown timeouts are coordinated - move EnvVarForProxy from config to packagemanager (it is package-manager knowledge); the shared function now builds the proxy URL and NO_PROXY itself, removing the duplicated construction in the per-command and persistent paths - relocate the #319 yarn and #339 IPv6 regression tests alongside the function - enable cloud sync in the persistent-proxy E2E workflow and fix the stale internal/proxystate path filter

Consistent timeout naming: *LockWait is the lock-acquire bound, *Timeout is the sync-RPC bound. Previously the final-flush pair was cloudFlushLockTimeout vs cloudFlushTimeout — two lookalike names for different operations.

The shutdown's final-flush block is now a cloudFlush helper, symmetric with startCloudSyncLoop (one-shot vs loop). Removed the triplicated ticker/lock contention comments, keeping the contract on the function doc and one-line pointers at the call sites.

The daemon now owns cloud delivery (periodic sync while serving + final flush on shutdown); stop signals it, waits, and reports the result. Rewrite the Cloud event sync section, fix stop attributions, add the cloud_sync state field, and update the sequence diagram.

Put the copy-paste recipes near the top so users find them before the internals.

- configurable bind host/port via --host/--port flags + config (listen_host, listen_port), bound directly to config fields per PMG's flag pattern - daemon log path via --log-file and readiness timeout in ProxyDaemonConfig; Daemonize no longer owns path policy (caller validates, fails fast) - gate periodic cloud sync on auto_sync; suppress detached background sync for proxy commands instead of flipping the flag - pmg proxy env --export emits shell-quoted lines for eval (spaces survive) - extract shared flows.BuildCachedMalysisAnalyzer, dropping the analyzer+cache duplication between proxy flow and proxy server - add internal/proxyserver/doc.go documenting the package + boundary vs flows - E2E: assert malicious installs are blocked (drop continue-on-error) - docs: trim Commands/State-file to user contracts; refresh bind address

- gate the shutdown cloud flush on auto_sync too, matching the periodic ticker (auto_sync consistently controls all daemon-driven cloud delivery) - ResolveStatePath takes cacheDir instead of *RuntimeConfig, keeping state.go free of config dependency - drop the empty-host comment in listenAddr; keep the loopback guard so a blank host never silently binds all interfaces

Sahilb315 added 5 commits June 24, 2026 16:28

refactor(flows): extract SetupCACertificate for reuse

5e3a806

Move the CA load/generate/merge logic out of proxyFlow into an exported flows.SetupCACertificate so the persistent proxy server can reuse it.

feat(action): add server-mode for persistent proxy

a6a7e7d

When server-mode=true the action starts the proxy daemon and injects proxy env vars into the job instead of installing shims.

test(proxy): add persistent proxy server E2E workflow

8a7c758

docs(readme): document persistent proxy server mode

bd6c80a

Sahilb315 mentioned this pull request Jun 24, 2026

feat(proxy): persistent proxy server (POC) #347

Closed

Sahilb315 added 5 commits June 24, 2026 16:40

docs: add persistent proxy server architecture doc

fc8adbd

refactor proxyserver

5067038

Sahilb315 marked this pull request as ready for review June 24, 2026 13:58

abhisek reviewed Jun 24, 2026

View reviewed changes

Comment thread cmd/proxy/env.go Outdated

Comment thread cmd/proxy/stop.go Outdated

Comment thread cmd/proxy/start.go Outdated

abhisek reviewed Jun 24, 2026

View reviewed changes

Comment thread internal/proxyserver/cloud_flush.go Outdated

abhisek reviewed Jun 24, 2026

View reviewed changes

Comment thread internal/proxyserver/cloud_flush.go Outdated

Comment thread internal/proxyserver/daemonize_unix.go Outdated

Sahilb315 added 4 commits June 24, 2026 21:50

disable auto-sync for proxy cmds

780a79f

Sahilb315 changed the title ~~feat(proxy): persistent proxy server mode (MVP)~~ feat(proxy): persistent proxy server mode Jun 24, 2026

Sahilb315 added 5 commits June 25, 2026 12:55

docs: move Usage section up below How it works

63dfeb0

Put the copy-paste recipes near the top so users find them before the internals.

abhisek reviewed Jun 25, 2026

View reviewed changes

Comment thread docs/persistent-proxy.md Outdated

Comment thread docs/persistent-proxy.md Outdated

Sahilb315 and others added 5 commits June 25, 2026 18:42

fix: Decouple localdb with malysis analyser construction

ac3ec42

fix: Persist global args before proxy server daemon exec

ed1ca2b

fix: GitHub Action for cloud auto-sync in server mode

e6085fc

abhisek approved these changes Jun 26, 2026

View reviewed changes

sudhanshutech approved these changes Jun 26, 2026

View reviewed changes

sudhanshutech merged commit c47776d into main Jun 26, 2026
23 checks passed

sudhanshutech deleted the feat/persistent-proxy-mvp branch June 26, 2026 05:49

Uh oh!

Conversation

Sahilb315 commented Jun 24, 2026

What this adds

Highlights

Structure

Deferred (separate work)

Notes for review

Uh oh!

codecov-commenter commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

safedep Bot commented Jun 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

SafeDep Report Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov-commenter commented Jun 24, 2026 •

edited

Loading

safedep Bot commented Jun 24, 2026 •

edited

Loading