🏥 CI Failureexamples test workflow: all curl commands fail with ssl connection error after pr #524

[github-actions]

Summary

The Examples Test workflow is experiencing a complete failure where all curl-based tests fail with exit code 35 (CURLE_SSL_CONNECT_ERROR) after merging PR #524.

Failed Run Details

• Run: 21726883581
• Workflow: Examples Test
• Commit: 54fd26e
• PR: fix: remove HTTP_PROXY/HTTPS_PROXY env vars from agent container #524 - "fix: remove HTTP_PROXY/HTTPS_PROXY env vars from agent container"
• Date: 2026-02-05T20:13:44Z

Failure Pattern

ALL curl tests failing with exit code 35:

1. ✅ basic-curl.sh setup - containers start successfully
2. ❌ basic-curl.sh execution - curl -s https://api.github.com → exit code 35
3. ✅ using-domains-file.sh setup - containers start successfully
4. ❌ using-domains-file.sh execution - curl -s https://api.github.com → exit code 35
5. ✅ debugging.sh setup - containers start successfully
6. ❌ debugging.sh execution - curl -s https://api.github.com/zen → exit code 35
7. ⏭️ blocked-domains.sh - skipped due to previous failures

Key Observations

✅ What Works

• Container creation and startup
• Squid healthcheck passes
• iptables rules applied successfully
• Network configuration (172.30.0.0/24)
• Docker Compose orchestration

❌ What Fails

• curl SSL/TLS connection to HTTPS endpoints
• 100% failure rate across all example tests
• Deterministic failure (not intermittent)

🔍 Container Logs Analysis

From the debugging.sh test logs:

[entrypoint] Proxy configuration:
[entrypoint]   HTTP_PROXY=
[entrypoint]   HTTPS_PROXY=
[entrypoint] Network information:
[entrypoint]   IP address: 172.30.0.20 
[entrypoint]   Hostname: 83c0d2a68e4f
[entrypoint] Executing command: /bin/bash -c curl -s https://api.github.com/zen

[DEBUG] Agent exit code: 35
``````

## Root Cause Analysis

### Curl Exit Code 35 Meaning

From curl documentation:
``````
CURLE_SSL_CONNECT_ERROR (35)
  A problem occurred somewhere in the SSL/TLS handshake.
  You really want the error buffer and read the message there
  as it pinpoints the problem slightly more.
``````

### PR #524 Changes

The merged PR removed HTTP_PROXY/HTTPS_PROXY environment variables:

**Rationale from PR**:
- "Intercept mode (iptables DNAT 80/443 → squid:3129) handles all routing transparently"
- "Port 3128 is unreachable from the agent container, causing Codex (Rust/reqwest) to fail"

**Changes made**:
1. Removed `HTTP_PROXY` and `HTTPS_PROXY` from agent container environment
2. Added proxy vars to `EXCLUDED_ENV_VARS` to prevent leaking via `--env-all`
3. Updated entrypoint.sh logging to show empty proxy vars

### Hypothesis

The SSL connection failure suggests one of these issues:

1. **iptables DNAT not working correctly**: Traffic to port 443 may not be redirecting to Squid's intercept port (3129)
2. **Squid intercept mode misconfiguration**: Squid may not be properly handling intercepted HTTPS (CONNECT) traffic
3. **Certificate verification issue**: curl in the agent container may not trust the connection without explicit proxy env vars
4. **Squid → External SSL handshake failure**: Squid may be failing to establish the outbound SSL connection

## Evidence Timeline

``````
20:14:47.6048977Z  Container awf-agent  Started
20:14:47.6247851Z [entrypoint] Agentic Workflow Firewall - Agent Container
20:14:47.6251074Z [iptables] NOTE: Host-level DOCKER-USER chain handles egress filtering
20:14:47.6252390Z [iptables] Squid proxy: squid-proxy:3128 (intercept: 3129)
20:14:47.6733845Z [iptables] Redirect HTTP (80) and HTTPS (443) to Squid intercept port...
20:14:47.6906418Z [iptables] NAT rules applied successfully
20:14:47.7058208Z [entrypoint] Executing command: /bin/bash -c curl -s https://api.github.com/zen
20:14:47.8345512Z [DEBUG] Agent exit code: 35

Only 0.1 seconds between command execution and failure - suggests immediate SSL handshake failure.

Impact

• 🔴 CRITICAL - All example tests blocked
• Cannot verify basic firewall functionality
• CI pipeline broken for main branch
• User-facing examples do not work

Recommended Investigation Steps

1. Check Squid access logs for connection attempts:

   sudo cat /tmp/squid-logs-1770322481586/access.log

2. Verify iptables DNAT rules are redirecting 443 traffic:

   docker exec awf-agent iptables -t nat -L -n -v

3. Test curl with verbose output to see SSL handshake details:

   sudo awf --allow-domains api.github.com --keep-containers -- curl -v https://api.github.com/zen

4. Compare with pre-PR fix: remove HTTP_PROXY/HTTPS_PROXY env vars from agent container #524 behavior: Check if containers built from parent commit (769a6f5) work correctly

5. Verify Squid intercept port configuration:

   docker exec awf-squid grep -E '(http_port|ssl_bump)' /etc/squid/squid.conf

Related Issues

• PR fix: remove HTTP_PROXY/HTTPS_PROXY env vars from agent container #524 - The change that introduced this regression
• Issue Codex smoke test fails: HTTP_PROXY points to port 3128 but explicit proxy is unreachable from agent container #523 - Original issue that PR fix: remove HTTP_PROXY/HTTPS_PROXY env vars from agent container #524 was trying to fix (Codex/Rust connection refused)
• Issue 🏥 CI Failuresquid container intermittent startup failure in debugging.sh test #505 - Previous Squid startup failures (different failure mode)

Files to Review

• containers/agent/setup-iptables.sh - NAT redirection rules
• src/docker-manager.ts - Container environment configuration
• src/squid-config.ts - Squid proxy configuration
• containers/squid/squid.conf - Squid intercept port setup

---

🏥 Automatically investigated by CI Doctor

AI generated by CI Doctor

[github-actions]

Recurring Failure - Run 21727992346

Commit: 995a2ea (PR #527)
Date: Feb 5, 2026 20:52 UTC

The SSL connection failure (exit code 35) continues to occur after PR #527 was merged. This confirms the issue is not resolved and is blocking the Examples Test workflow.

Failure Pattern

Test debugging.sh failed with the same symptoms:

• ✅ Containers started successfully
• ✅ Squid healthcheck passed
• ✅ iptables rules configured
• ❌ Curl command fails: curl -s https://api.github.com/zen → exit code 35

Impact

ALL curl-based example tests are affected:

• basic-curl.sh ❌
• using-domains-file.sh ❌
• debugging.sh ❌
• blocked-domains.sh (skipped due to earlier failure)

Next Steps

The Examples Test workflow is completely broken. Priority investigation needed:

1. Check Squid logs - Are CONNECT requests reaching Squid?
2. Verify iptables DNAT - Is HTTPS traffic (port 443) being redirected?
3. Test without firewall - Does curl work with direct proxy environment vars?
4. SSL/TLS debugging - Add curl -v to see handshake details

---

🤖 Automatically detected by CI Doctor from workflow run 21727992346

AI generated by CI Doctor

[github-actions]

🍪 Issue Monster has assigned this to Copilot!

I've identified this issue as a good candidate for automated resolution and assigned it to the Copilot agent.

The Copilot agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

AI generated by Issue Monster