Skip to content

Honor runner proxy settings in CLI proxy and MCP gateway containers - #70347

Open
pelikhan with Copilot wants to merge 4 commits into
mainfrom
copilot/fix-http-proxy-issues
Open

pelikhan with Copilot wants to merge 4 commits into
mainfrom
copilot/fix-http-proxy-issues

Conversation

Copilot AI commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

On proxy-only self-hosted runners, the CLI proxy container lacks the runner’s proxy settings. Local health checks pass, but upstream GitHub API requests return HTTP 502 and prevent agent startup.

  • Proxy forwarding: Pass defined HTTP_PROXY, HTTPS_PROXY, and NO_PROXY variables, including lowercase forms, to both containers by name. Preserve Docker defaults for unset variables and explicit gateway overrides.
  • Startup validation: Probe the upstream API through the CLI proxy before reporting readiness. Bound localhost requests, bypass the runner’s HTTP proxy, and remove the container on failure.
  • Credential protection: Redact proxy environment values from CLI proxy and gateway diagnostics.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix HTTP(S)_PROXY/NO_PROXY passing to awmg-cli-proxy Honor runner proxy settings in CLI proxy and MCP gateway containers Oct 19, 2026
Copilot AI requested a review from pelikhan October 19, 2026 19:53
@pelikhan
pelikhan marked this pull request as ready for review October 19, 2026 20:10
Copilot AI max review requested due to automatic review settings October 19, 2026 20:10
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · gpt56 · 8.04 AIC · ⌖ 5.49 AIC · ⊞ 14.7K · ◷

Generated by Ponytail Reviewer for #70347

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

Design Decision Gate 🏗️

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Design Decision Gate 🏗️ · opus55 · 10.2 AIC · ⌖ 31.9 AIC · ◇ 13.7 AIC · ⊞ 1.9K · ◷

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · sonnet50 · 13.2 AIC · ⌖ 15.1 AIC · ◇ 6.61 AIC · ⊞ 12.6K · ◷

🧪 Test quality analysis by Test Quality Sentinel

@github-actions

Copy link
Copy Markdown
Contributor

🔎 PR Code Quality Reviewer is reviewing code quality for this pull request...

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd (via pr-triage) on this proxy-forwarding fix. The change is well-tested and correctly addresses the root cause (#70266) — only minor maintainability suggestions below, no blocking issues.

📋 Key Themes & Highlights

Key Themes

  • Duplicated proxy variable list: the six-name proxy env list (HTTP_PROXY/HTTPS_PROXY/NO_PROXY + lowercase) is copy-pasted across start_cli_proxy.sh, redact_cli_proxy_diagnostics.cjs, and start_mcp_gateway.cjs, risking silent drift if a future variable (e.g. ALL_PROXY) is added to only some of them.
  • Composition invariant untested: injectCustomGatewayEnvArgs and injectProxyEnvArgs are unit-tested individually, but their ordering invariant when composed in main() (name-only proxy flags must precede explicit custom env values so explicit wins) has no direct test.

Positive Highlights

  • ✅ Proxy values are forwarded by name only (-e NAME), never embedded in Docker argument strings or diagnostics — solid root-cause fix for the credential-leak and injection-risk classes this area is sensitive to.
  • ✅ New upstream /api/v3/rate_limit probe correctly distinguishes "local proxy up" from "upstream reachable through proxy", which is exactly the gap described in #70266 (local health checks passing while upstream requests still 502).
  • ✅ Good test coverage: parameterized tests for each proxy var name, empty-value preservation, unset-defaults-to-Docker-client behavior, and the new upstream-failure path (container cleanup + exit code).
  • ✅ --noproxy '*' --connect-timeout --max-time bounds the local health/upstream probes so they can't hang or get routed back through the runner's own proxy.

Posted 2 inline suggestions (both non-blocking) on the duplicated constant list and the missing composition test.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 120.2 AIC · ⌖ 15.1 AIC · ⊞ 10.1K
Comment /matt to run again

Comment thread actions/setup/sh/start_cli_proxy.sh Outdated
fi

PROXY_ENV_ARGS=()
for proxy_var in HTTP_PROXY HTTPS_PROXY NO_PROXY http_proxy https_proxy no_proxy; do

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/codebase-design] The proxy variable name list (HTTP_PROXY HTTPS_PROXY NO_PROXY http_proxy https_proxy no_proxy) is now duplicated in three places: this loop, redact_cli_proxy_diagnostics.cjs:16, and start_mcp_gateway.cjs:506. Nothing enforces they stay in sync.

💡 Suggested fix

Extract a single source of truth — e.g. a PROXY_ENV_NAMES constant in constants.cjs shared by both JS files, and a comment in the shell script cross-referencing it as the canonical list. If a future proxy variable (e.g. ALL_PROXY) is added, it's easy to update only one or two of the three copies and silently leave a redaction or forwarding gap.

@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in d4319bc: proxy variable names now come from the shared actions/setup/sh/proxy_env_names.list, consumed by the shell and both JavaScript paths.

Comment thread actions/setup/js/start_mcp_gateway.cjs Outdated
return;
}
args = injectCustomGatewayEnvArgs(args);
args = injectProxyEnvArgs(args);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] injectCustomGatewayEnvArgs and injectProxyEnvArgs are each unit-tested in isolation, but main() composes them sequentially on the same args array and relies on an ordering invariant (proxy name-only flags must precede explicit custom -e NAME=value flags so the explicit value wins). No test exercises both functions together.

💡 Suggested test
it('keeps explicit custom env values winning over inherited proxy forwarding', () => {
  const args = injectCustomGatewayEnvArgs(['run', '__CUSTOM_ENV__', 'gateway:latest'], {
    GH_AW_MCP_GATEWAY_CUSTOM_ENV_NAMES: JSON.stringify(['HTTP_PROXY']),
    GH_AW_MCP_GATEWAY_CUSTOM_ENV_0: 'explicit-value',
  });
  const result = injectProxyEnvArgs(args, { HTTP_PROXY: 'inherited-value' });
  expect(result.indexOf('HTTP_PROXY')).toBeLessThan(result.indexOf('HTTP_PROXY=explicit-value'));
});

This pins down the real invariant the two functions depend on, rather than trusting it holds because each function is independently correct.

@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in d4319bc: added a composition test confirming inherited name-only proxy flags precede explicit custom gateway -e NAME=value flags.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved credential exposure and local MCP connectivity regressions block approval.

3 open issues
What changed in this PR

Updates gh-aw setup to support proxy-only self-hosted runners.

Changes:

  • Forward defined runner proxy variables to both containers.
  • Check upstream API access before reporting CLI proxy readiness.
  • Extend diagnostic redaction and regression coverage.
File Description
actions/​setup/​sh/​start_cli_proxy.sh Forwards proxy settings and validates upstream connectivity.
actions/​setup/​js/​start_mcp_gateway.test.cjs Tests forwarding, overrides, redaction, and startup ordering.
actions/​setup/​js/​start_mcp_gateway.cjs Injects proxy variables and extends diagnostic sanitization.
actions/​setup/​js/​start_cli_proxy.test.cjs Covers proxy forwarding and upstream startup failures.
actions/​setup/​js/​redact_cli_proxy_diagnostics.cjs Adds proxy environment values to diagnostic masking.

Review effort: Max

Comment thread actions/setup/js/start_mcp_gateway.cjs Outdated
redacted = redacted.replace(pattern, replacement);
}
return redacted;
return redactCliProxyDiagnostics(redacted, env);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in d4319bc: generic credential-pattern spans and registered environment values are matched against the original diagnostics and coalesced before replacement. Regression tests cover both overlap directions.

Comment thread actions/setup/js/start_mcp_gateway.cjs Outdated
Comment on lines +507 to +509
const proxyArgs = names.filter(name => env[name] !== undefined).flatMap(name => ["-e", name]);
// Insert after "run" so explicitly configured container settings still win.
return [...args.slice(0, 1), ...proxyArgs, ...args.slice(1)];

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in d4319bc: defined NO_PROXY/no_proxy values now retain existing exclusions and add host.docker.internal plus awf-enclave-mcp; unset Docker defaults and explicit gateway overrides remain intact.

Comment thread actions/setup/sh/start_cli_proxy.sh Outdated
Comment on lines +181 to +182
if ! curl -sf --noproxy '*' --connect-timeout 5 --max-time 15 \
--cacert "$TLS_DIR/ca.crt" "https://localhost:18443/api/v3/rate_limit" -o /dev/null 2>/dev/null; then

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in d4319bc: upstream readiness now makes up to three bounded probes before cleanup; a regression test confirms startup succeeds when the second probe recovers.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot address the following outstanding work in one pass:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
  2. Review (actions/setup/sh/start_cli_proxy.sh:132): [/codebase-design] The proxy variable name list (HTTP_PROXY HTTPS_PROXY NO_PROXY http_proxy https_proxy no_proxy) is now duplicated in three places: this loop, redact_cli_proxy_diagnostics.cjs:16, and start_mcp_gateway.cjs:506. Nothing enforces they stay in sync. - Honor runner proxy settings in CLI proxy and MCP gateway containers #70347 (comment)
  3. Review (actions/setup/js/start_mcp_gateway.cjs:1046): [/tdd] injectCustomGatewayEnvArgs and injectProxyEnvArgs are each unit-tested in isolation, but main() composes them sequentially on the same args array and relies on an ordering invariant (proxy name-only flags must precede explicit custom -e NAME=value flags so the explicit value wins). No test exercises both functions together. - Honor runner proxy settings in CLI proxy and MCP gateway containers #70347 (comment)
  4. Review (actions/setup/js/start_mcp_gateway.cjs:408): The generic credential regexes can fragment a registered proxy URL before exact-value masking. With HTTPS_PROXY=(token/redacted) diagnostics become (token/redacted) exposing part of the proxy password in workflow logs. Match both credential patterns and environment values against the original text, then coalesce overlaps before replacing them, as redactArtifactContent does. Simply reversing the passes can instead hide credential keys when an environment value overlaps them. Add regression tests for both overlap directions. - Honor runner proxy settings in CLI proxy and MCP gateway containers #70347 (comment)
  5. Review (actions/setup/js/start_mcp_gateway.cjs:509): Forwarding the runner's proxy unchanged also affects local HTTP MCP backends. The compiler uses host.docker.internal for mcp-scripts and rewritten localhost URLs, and awf-enclave-mcp for the enclave backend. The gateway's Go HTTP transport honors these proxy variables; its automatic loopback bypass does not cover those hostnames, even when Docker maps one to loopback. With HTTP_PROXY set and NO_PROXY=localhost,127.0.0.1, these requests therefore go through the upstream proxy, which cannot reach the runner/container-local backends, and gateway startup can fail. Add effective bypasses for generated internal backend hosts while preserving existing exclusions, Docker defaults, and explicit gateway overrides. Cover this proxy-plus-local-backend configuration in tests. - Honor runner proxy settings in CLI proxy and MCP gateway containers #70347 (comment)
  6. Review (actions/setup/sh/start_cli_proxy.sh:182): This upstream readiness gate makes only one attempt. A transient API/proxy 502 or 503, or a timeout during the first proxy CONNECT, now removes the container and fails the job before downstream startup retries can help. Retry transient failures a small, bounded number of times before cleanup, keeping the existing per-request timeouts and localhost proxy bypass. Add a regression case where the first upstream probe fails and a subsequent attempt succeeds. - Honor runner proxy settings in CLI proxy and MCP gateway containers #70347 (comment)
  7. Fix failing check agent (FAILURE): https://github.com/github/gh-aw/actions/runs/39317132977/job/118282635106.
  8. Fix failing check conclusion (FAILURE): https://github.com/github/gh-aw/actions/runs/39317132977/job/118284692849.
  9. Fix failing check JS Tests (shard 2/4) (FAILURE): https://github.com/github/gh-aw/actions/runs/39317133024/job/118281778469.

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: 8e87708
Sous-chef work: 043185828d65a534dd8b2b6b4d00e0d47bf21a2940d3f277c0ed15da7f668648 34f38971738f6a752a646bd7797a6c8d47fb096e4eb6f91e946a91255433c8c4 51e2287c3d1ee41b7e13769862add61dcd292a623cf943abc5968c984e246bcf 7fecc21e592fba2b2ac6c2136f84cd4ae3aabc119b843072a64b2e47812883a6 86e92527287fec4cf7a9c33e4549a272ae5d5530439eea46cb2334bc06db93c5 b29aaf3b14fb2db020180c80687aa7a4fe397d043067edaf822d861fe1b30f01 bcacfcfdea89f9dc1618f1e4e2680f433e91e2cbb5e15a460380cf171b16e652 f6f8ad41891b239c4633d626a38648da5dff450ad75cdeb90b742436c6347ad1
Sous-chef state: 255f058a0a974c246ca423ac56a869c424c9e427de7f19bce916e9fd773329a9

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 21.2 AIC · ⌖ 8.31 AIC · ⊞ 1.1K · ◷
Comment /souschef to run again

Copilot AI and others added 2 commits October 19, 2026 21:02
…y-issues

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
@pelikhan

Copy link
Copy Markdown
Collaborator

@lpcox review

@lpcox

lpcox commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

There is one remaining correctness issue in actions/setup/js/start_mcp_gateway.cjs:510–511: injectProxyEnvArgs only adds the internal-host exclusions when NO_PROXY or no_proxy already exists. If the runner sets HTTP_PROXY but neither exclusion variable (and Docker supplies no exclusions), the gateway forwards requests for host.docker.internal and awf-enclave-mcp through the corporate proxy, which cannot reach those internal MCP backends. Gateway startup can therefore still fail.

I reproduced this using the helper from head d4319bc6e5958c9ba3723c2d9db05e562f62b68c and Go’s http.ProxyFromEnvironment: with only HTTP_PROXY=http://proxy.example:8080, both internal URLs select that proxy; with an existing NO_PROXY, the helper appends the internal hosts and both URLs correctly bypass it.

Please ensure effective internal-host exclusions whenever proxy forwarding is enabled, including when both NO_PROXY forms are unset, while preserving existing exclusions, Docker defaults, and explicit gateway overrides. Add a regression case for an HTTP_PROXY-only environment.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

start_cli_proxy.sh does not pass the runner's HTTP(S)_PROXY/NO_PROXY to awmg-cli-proxy (HTTP 502 on proxied self-hosted runners)

5 participants