feat(auth): Phase 2 — AWS Sigv4, GCP IAP, Azure AD providers (v0.11.0)

[naveen-kurra]

Summary

Phase 2 of the pluggable auth provider work — three cloud-native providers
on top of the Phase 1 foundation (#77 / 7998f12). Customers authenticate
to Forge using identities they already have in their cloud; no parallel
IdP required.

Provider	What it does	Client token format
aws_sigv4	Caller mints a pre-signed STS GetCallerIdentity URL with their AWS SDK; Forge invokes it. STS returns the caller's canonical ARN. Same pattern as aws-iam-authenticator (EKS).	Authorization: Bearer forge-aws-v1.<base64-of-presigned-sts-url>
gcp_iap	Verify the JWT GCP IAP forwards as X-Goog-Iap-Jwt-Assertion when Forge sits behind a GCP HTTPS LB + IAP.	X-Goog-Iap-Jwt-Assertion: <jwt>
azure_ad	Verify Microsoft Entra ID Bearer tokens with tenant lock-in and optional Microsoft Graph group enrichment. Composes the Phase 1 oidc provider.	Authorization: Bearer <aad-jwt>

Forge never holds any IdP secrets — all three providers verify a caller-
minted credential against a third party (STS / GCP JWKS / AAD JWKS).

Why this matters

Today, putting Forge behind any of the three big cloud IdPs requires
standing up a parallel OIDC issuer (Cognito for AWS, Workspace SAML, etc.).
This PR removes that friction:

• AWS: any Lambda / EC2 / EKS / IAM user calls Forge using their
  existing IAM credentials. Zero secrets stored on Forge, no token endpoint
  to host.
• GCP: Forge sitting behind GCP LB+IAP consumes IAP's signed user
  assertion directly.
• Azure: Entra tokens (humans, CI, Azure workloads) work natively with
  AAD-specific quirks (tenant gate, groups overage) handled correctly.

Design pivot during PR review — aws_sigv4 switched to pre-signed URL pattern

The PR went through an in-flight design correction caught by real-AWS
smoke testing. The TL;DR:

What was wrong (original design — commits 9a1ebae through 382294e)

The first design had clients sign their POST to Forge using AWS Sigv4
("header reflection"). Forge would forward the signed headers to STS,
expecting STS to validate them.

This is broken in a deterministic way: Sigv4 binds its signature to
the destination host as part of the canonicalized signing input.
Headers signed for Forge's hostname can't be replayed against STS —
STS computes host: sts.<region>.amazonaws.com, recomputes the
signature, gets a different hash, and rejects with
SignatureDoesNotMatch.

Standard tools (awscurl, boto3.client('sts'), all AWS SDKs) always
sign for the URL they're calling, so there was no working client path.

What's correct (current design — commit 8568535)

Switched to the pre-signed URL pattern that aws-iam-authenticator
uses for EKS:

1. Caller's AWS SDK mints a pre-signed GetCallerIdentity URL — signature
   embedded in query params, signed for STS's host.
2. Caller base64-encodes the URL, prepends forge-aws-v1., sends as a
   standard Authorization: Bearer … header.
3. Forge decodes, validates the URL host matches sts.<region>.amazonaws.com
   (SSRF guard), GETs the URL, parses the XML response, stamps Identity.

Why this is the right design (not just a fix)

Three design properties land cleanly:

1. Client UX matches OIDC/JWT/Azure AD. All three Phase 2 providers
   now have the same caller experience: mint a Bearer token, attach it,
   send. Three lines of Python (or any AWS-SDK-bearing language). No
   custom signing logic, no header manipulation, no per-call procedure.
2. No re-engineering of standard tools. aws-iam-authenticator-style
   token format works with any AWS SDK's SigV4QueryAuth API.
3. Server is simpler. Forge doesn't reflect headers; it just GETs a
   URL. ~80 LOC of STS client code instead of header-forwarding plumbing.

Locked design decisions (unchanged through the pivot)

• §9.1 — No aws-sdk-go-v2 dependency. STS client is hand-rolled HTTP + XML.
• §9.2 — azure_ad composes Phase 1 oidc.Provider; no JWT/JWKS code in azure_ad/.
• §9.3 — allowed_principals uses path.Match shell globs (no regex).
• §9.4 — IAP issuer + JWKS URL hardcoded; no operator-overridable knob.

What clients write (the actual 3 lines)

import boto3, base64
from botocore.auth import SigV4QueryAuth
from botocore.awsrequest import AWSRequest

creds = boto3.Session().get_credentials().get_frozen_credentials()
req = AWSRequest(method='GET',
                 url='https://sts.us-east-1.amazonaws.com/?Action=GetCallerIdentity&Version=2011-06-15')
SigV4QueryAuth(creds, 'sts', 'us-east-1', expires=900).add_auth(req)
token = 'forge-aws-v1.' + base64.urlsafe_b64encode(req.url.encode()).rstrip(b'=').decode()

requests.post(forge_url, headers={'Authorization': f'Bearer {token}'}, data=msg)

A reference client lives at scripts/forge-aws-sign.py (~80 LOC; CLI flags
for --region, --profile, --token-only, etc.).

Note on boto3: the high-level boto3.client('sts').generate_presigned_url('get_caller_identity', ...) returns a URL STS rejects with SignatureDoesNotMatch even on direct curl — known boto3 limitation (it signs as if the request were a POST). The reference client uses the lower-level SigV4QueryAuth API directly, same as aws-iam-authenticator does internally.

✅ Real-AWS validation (live AWS account)

End-to-end validated against real AWS STS using SSO assumed-role
credentials in account 412664885516. Forge instance built from
commit 8568535, running locally.

#	Scenario	Expected	Actual	Validates
1	curl with no auth headers	401, missing_token	✅ 401 + valid bearer token required	Phase 1 middleware enforces auth; review #4 reason-code preserved
2	Pre-signed URL token, empty allowlist	200 / handler-error, auth_verify fires	✅ HTTP 400 (body shape; auth path complete), auth_verify emitted	STS reflection works end-to-end against real STS
3	Same token, matching allowed_principals	200 / handler-error (auth pass)	✅ HTTP 400, auth_verify fires	ArnMatcher accepts matching ARNs
4	Same token, wrong-account allowed_principals	401, rejected	✅ 401 + token rejected by auth provider	ArnMatcher correctly denies non-matching ARNs

Audit log emitted (Test #2 success path)

{
  "event":"auth_verify",
  "fields":{
    "method":"POST",
    "path":"/tasks/send",
    "provider":"aws_sigv4",
    "user_id":"arn:aws:sts::412664885516:assumed-role/AWSReservedSSO_PowerUserAccess_c794d5f2c2fe4370/Naveen",
    "org_id":"412664885516",
    "token_kind":"sigv4",
    "groups_count":0,
    "remote_addr":"[::1]:62448"
  }
}

Every field correct: provider matches the configured name, user_id is the STS-returned canonical ARN (assumed-role form, including session name), org_id is the AWS account number, token_kind is the new forge-aws-v1.-prefix detection.

Two bugs caught during live testing (both fixed in 8568535)

1. URL re-encoding through net/url. Round-tripping the presigned URL through Go's *url.URL.String() re-encoded query params in ways that differed from how the AWS SDK emitted them (e.g., / in X-Amz-Credential, + inside X-Amz-Security-Token). STS recomputed the canonical request from those re-encoded bytes and rejected. Fix: PresignedToken keeps a RawURL string field with the byte-for-byte original; the parsed *url.URL is used only for SSRF host validation and query-param inspection.
2. boto3's generate_presigned_url quirk. The reference client originally used boto3.client('sts').generate_presigned_url('get_caller_identity', …), which produces a URL STS rejects with SignatureDoesNotMatch (known boto3 quirk — signs as if POST). Fix: the reference client now uses the lower-level SigV4QueryAuth.add_auth() directly, same pattern aws-iam-authenticator uses.

Other layers also exercised live

• ✅ Egress allowlist auto-extended to include sts.us-east-1.amazonaws.com — Forge's outbound STS call was permitted.
• ✅ Loopback static_token still works for the dashboard at http://localhost:9999/ (auto-prepended via PrependChain — Phase 1 review Add per-agent secrets, build signing, and forge framework #10 invariant intact under Phase 2).
• ✅ Config hot-reload picks up forge.yaml content changes for most fields. Caveat: auth-chain providers are constructed once at startup; modifying allowed_principals requires a hard restart (Ctrl-C + forge run again) for the new allowlist to take effect. Documented as a follow-up — affects all providers, not just aws_sigv4.
• ✅ Auth-before-Egress wizard ordering (639bfa9) ensures wizard-scaffolded configs include the STS host in egress_hosts.

What was NOT live-tested (and why)

• gcp_iap provider — requires a GCP project with HTTPS LB + IAP enabled. Covered by unit tests with a fake JWKS signer.
• azure_ad provider — requires an Entra tenant + app registration. Covered by unit tests with a fake AAD.

Per PHASE2_TEST_STRATEGY.md §8.2, those live tests run at release-tag time, not on every PR.

What landed (12 commits)

#	Commit	Theme
PR 1	55942d8	Header contract extension (foundation)
PR 2	9a1ebae	aws_sigv4 provider (initial — header-reflection design)
PR 3	5b71071	gcp_iap provider
PR 4	1e23140	azure_ad provider
PR 5	47c4748	Wizard + non-interactive flags + Web UI integration
PR 6	98578f9	Operator docs + CHANGELOG
fix	639bfa9	Auth step runs before Egress; auth hosts auto-added to allowlist
audit	b5f303b	Audit-pass refinements (iap_jwt token_kind, URL parse caching, cleanup)
docs	aaf8375	Gitignore docs/auth (per reviewer feedback)
client	382294e	Reference client + "client signing contract" docstring (original pattern)
pivot	b3444c2	Switch aws_sigv4 to pre-signed URL pattern
live	8568535	Live-test fixes: preserve raw URL; use SigV4QueryAuth in client

Total: ~42 files, +5,500 / -700 lines net (mostly tests + the design-pivot rewrite).

Phase 1 compatibility

• Zero forge.yaml changes required for callers using Phase 1 providers (static_token, oidc, http_verifier). Phase 1 test suite passes unmodified.
• The auth Headers map gained one new key — X-Goog-Iap-Jwt-Assertion for gcp_iap. Existing keys unchanged.
• The oidc package gained an internal SkipIssuerCheck field with yaml:"-" — unreachable from forge.yaml, only set by azure_ad multi-tenant. Operators see no change.

Security model highlights

• No IdP secret on Forge. STS / GCP / AAD do the cryptographic work; Forge reflects or verifies, never holds keys.
• SSRF guard on aws_sigv4 — the pre-signed URL host MUST match sts.<configured-region>.amazonaws.com. A token whose URL points elsewhere is rejected at parse time, before any outbound request.
• Algorithm whitelist before key lookup on the OIDC / IAP / AAD providers — algorithm-confusion attacks rejected before reaching JWKS.
• Tenant gate before Graph on azure_ad — multi-tenant requires explicit opt-in; Graph calls only fire after tid validation.
• Caller's Bearer reflected to Graph — Forge holds no Graph credentials.

Known deferred work

• TUI sub-step input flows for the three new providers — non-interactive flag path is the production-critical surface; TUI users pick "Custom" and edit forge.yaml directly until the TUI follow-up lands.
• Hot-reload of auth chain — config edits to auth.providers (incl. allowed_principals) require a hard forge run restart. Affects all providers, not just Phase 2.

Test plan

• Unit tests: happy path, malformed input, scope/tenant rejection, allowlist miss, cache hit/miss, stale-grace, soft-fail, alg confusion, body caps, redirect attacks — all three new providers
• Pre-signed URL parser fuzz test (corpus-seeded, runs in CI)
• Phase 1 regression suite passes unmodified
• go test -race -count=1 ./... — 42 packages green
• golangci-lint v2.10.1 — 0 issues
• gofmt -l forge-core forge-cli forge-plugins — clean
• Anti-pattern grep gates (no aws-sdk-go import, IAP constants confined, no JWT in azure_ad, skip_issuer_check never in YAML) — all pass
• Live AWS validation — happy + deny paths against real STS (see "Real-AWS validation" above)
• Manual smoke for gcp_iap and azure_ad — runs at release-tag time per PHASE2_TEST_STRATEGY.md §8.2

Design artifacts (offline)

Full design package in ~/Desktop/forge_designs_and_PRD/phase2_implementation/:

• PHASE2_CLOUD_NATIVE_PROVIDERS.md — top-level design + §9 locked decisions
• PHASE2_PROGRESS_MAP.md — diagram-to-PR tracker
• PHASE2_TEST_STRATEGY.md — pyramid, harnesses, security catalog, CI gates, manual smoke
• 7 PlantUML diagrams (component / class / startup / request flow / per-provider deep-dives)
• PR1_HEADER_CONTRACT.md through PR6_DOCS.md — per-PR checklists with code sketches and acceptance criteria

[initializ-mk]

Phase 2 review — three providers, focused on the outbound-HTTP defense-in-depth gaps

I ran a security-focused audit across the four surfaces: aws_sigv4, gcp_iap, azure_ad, and the middleware/CLI/UI wiring. Tests pass with -race across all 9 affected packages, golangci-lint clean across all three modules, gofmt clean. The crypto choices, sentinel-error contract, loopback prepend invariant, and SkipIssuerCheck YAML-unreachability all hold up.

The pattern in the BLOCKERs below is the same in three places: the outbound *http.Client is constructed with &http.Client{Timeout: …} and nothing else. That leaves Go's default redirect policy (follow up to 10), which silently undermines the SSRF / same-host guard that each provider does at the application layer. The repo already has security.SafeRedirectPolicy(10) and uses it in the webhook/http_request paths — wire it (or http.ErrUseLastResponse) into these three clients.

---

BLOCKERS (please fix before merge)

B1. azure_ad/graph_client.go:138 — ensureGraphHost accepts http:// nextLink
ensureGraphHost only compares got.Host; the scheme is not checked. A @odata.nextLink: \"http://graph.microsoft.com/...\" passes the gate. The next iteration sets Authorization: <caller's bearer> and sends it in plaintext. Go strips Authorization on cross-host redirects but NOT on cross-scheme downgrades to the same host. Fix: also require got.Scheme == \"https\" (or == c.endpointScheme so test mode still works).

B2. azure_ad/graph_client.go:51 — Graph client follows arbitrary HTTP redirects
&http.Client{Timeout: timeout} uses the default CheckRedirect. The application-layer ensureGraphHost only runs on @odata.nextLink (line 73), not on HTTP 301/302/307. A 302 from Graph (or a MITM with a cert that ever cracks) would silently follow, parse an attacker body, and yield to JSON unmarshal. Fix: set CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse } — Graph's nextLink mechanism makes HTTP redirects unnecessary.

B3. aws_sigv4/sts_client.go:33 — STS client follows arbitrary HTTP redirects
Same root cause. The parser-side host validation (sigv4_parser.go:88) only covers the first hop; once http.Client.Do follows a redirect off sts.<region>.amazonaws.com, an attacker-controlled body becomes the parsed STS XML and controls Identity.UserID / OrgID / Arn. The token-host gate is exactly the SSRF guard the design relies on; redirect-following silently invalidates it. Fix: pin CheckRedirect to ErrUseLastResponse on this client.

---

MAJOR (should land in this PR or a fast follow-up)

M1. aws_sigv4/sigv4_parser.go:88 — no rejection of URL userinfo
https://user:pass@sts.us-east-1.amazonaws.com/?... passes the EqualFold(u.Host, expectedHost) check (u.Host excludes userinfo per RFC 3986). Forge then ships Basic-auth bytes from an attacker-controlled token to STS. STS ignores it, but it's still attacker bytes leaving the box. Add if u.User != nil { return error }.

M2. aws_sigv4/sigv4_parser.go — no X-Amz-Date freshness or X-Amz-Expires sanity
STS enforces a ~15min window, but the cache key is hash(AKID|YYYYMMDD), so a stolen-token replay window relies entirely on STS's own enforcement. Reject X-Amz-Expires > 900 and an X-Amz-Date older than ~15min parser-side as belt-and-braces.

M3. azure_ad/provider.go:153 — multi-tenant has no allowed_tenants allow-list
When allow_multi_tenant: true, both the iss check (via SkipIssuerCheck) AND the tid check are suppressed — any Entra tenant in the world verifies. This is the documented trade-off, but the spec at resolveIssuer line 181 says "azure_ad enforces tenant via the tid claim instead," which is misleading: in multi-tenant mode there's no tid enforcement at all. Either add an allowed_tenants: []string knob, or fix the doc to be loud about "any tenant globally."

M4. forge-core/auth/middleware.go:110 — token_kind=\"iap_jwt\" only recorded when Bearer is missing
The kind := TokenKind(token) runs first; the IAP upgrade only triggers when kind == \"empty\". A request with a real Bearer JWT plus an IAP assertion records kind=\"jwt\", masking IAP-fronted traffic in audit dashboards. Audit-pipeline goal of counting IAP traffic distinctly is defeated. Fix: upgrade to iap_jwt whenever the IAP header is present AND the chain verifies via the gcp_iap provider.

M5. forge-core/validate/auth.go:64 — validateProviderSettings permissive on unknown keys
A typo like aud: instead of audience: slips through silently — asString(settings, \"audience\") returns "" and the required-keys check misses it. Combined with handlers_create.go forwarding Settings unfiltered into the on-disk scaffold, the Web UI POST {\"settings\": {\"audience\": \"x\", \"evil_key\": \"y\"}} produces a forge.yaml with evil_key in it. Today every field has yaml:\"-\" discipline so this is harmless — but it's one missed tag away from being exploitable. Add a closed-key whitelist + warn on unknown.

---

NITs / follow-ups (don't block merge)

• gcp_iap/iap_jwks.go:285 — classifyJWTErr substring matching is brittle. Prefer errors.Is(err, jwt.ErrTokenExpired) etc. via golang-jwt v5's named sentinels.
• gcp_iap/iap_jwks.go:213 — j.keys = keys unconditionally replaces during refresh. GCP rotates with overlap so today this is fine, but a partial-but-valid JWKS body (>=1 key but missing an in-flight kid) would drop kids the stale-grace contract assumes are kept. Consider merge-on-success or at least logging the diff.
• azure_ad/graph_cache.go:33 — Get returns the cached slice without copy; any caller that mutates id.Groups mutates the cache.
• forge-cli/cmd/init_auth.go:338 — needsYAMLQuoting catches all-digit (the 3364ca8 fix) but misses scientific-notation (1e10), hex (0x10), leading-zero (010), and .inf/.nan. Auth-setting values rarely look like these, but the docstring says "false negatives are bugs."
• aws_sigv4/identity_cache_test.go:69 — string(rune(i)) for surrogate code points yields \\ufffd and collides keys. Use strconv.Itoa(i) so the threshold test actually reaches 10,001.
• Several files: req, _ := http.NewRequestWithContext(...) discards the error. Hardcoded URLs make the panic risk currently zero, but it's an antipattern in security-critical code.
• gcp_iap/provider_test.go — add an HS256-with-EC-public-key-as-secret test case (most dangerous alg-confusion shape, currently only RS256 covered).

---

Verdict

Needs revision before merge — three BLOCKERs (B1–B3) are each 1–3 line CheckRedirect/scheme-check fixes plus tests. None require architectural change; all three undermine the application-layer SSRF / same-host guards that the design correctly identifies as the security boundary. The MAJORs (M1–M5) are higher signal-to-effort fixes and should land alongside.

What's not a concern:

• ✅ SkipIssuerCheck is genuinely unreachable from YAML (grep + behavior verified).
• ✅ Headers.Get is case-insensitive (forge-core/auth/provider.go:77); the lowercase-IAP-header concern is cleared.
• ✅ Loopback static_token PrependChain still wraps Phase 2 providers; localhost dashboard works.
• ✅ Phase 1 backward compat — zero forge.yaml changes required, Phase 1 tests pass unmodified.
• ✅ STS XML root-element enforcement (Go's encoding/xml honors the XMLName xml.Name tag).

Live AWS validation in the PR description is excellent — that level of evidence for the happy and deny paths is exactly the right bar. Once the three redirect fixes land, this is ready.

[initializ-mk]

@naveen-kurra also run /sync-docs command/plugin for Claude to sync the document updates based on the changes

[naveen-kurra]

@naveen-kurra also run /sync-docs command/plugin for Claude to sync the document updates based on the changes

DOne

[initializ-mk]

Re-review of fix commits — LGTM, ready to merge

Verified all 8 fix commits against the prior review. Tests pass with -race across 9 packages, lint + gofmt clean.

BLOCKERs — all cleared (774268d)

• B1 azure_ad/graph_client.go:166-181 — ensureGraphHost now takes configuredScheme and requires EqualFold(scheme) match. endpointScheme pre-parsed at construction. The http:// bearer-downgrade attack is closed.
• B2 azure_ad/graph_client.go:54-67 — CheckRedirect: func(...) error { return http.ErrUseLastResponse } with inline comment naming the threat model. Graph's app-layer nextLink replaces the need for HTTP redirects.
• B3 aws_sigv4/sts_client.go:39-48 — same CheckRedirect pinning with comment explicitly enumerating threats (MITM, TLS-inspecting proxy, DNS hijack).
• Bonus gcp_iap/iap_jwks.go — author proactively pinned the same on the JWKS client even though I didn't flag it. Same blast radius (attacker JWKS → forged tokens verify); correct defensive call.

MAJORs — all addressed

• M1 674bf2f — u.User != nil check added BEFORE the host check in sigv4_parser.go:88. TestParseToken_RejectsUserinfo pins it. Comment explains why STS ignores Basic auth but bytes still shouldn't leave the box.
• M2 c6e0aff — PresignedToken gains SigTime / Expires; parser requires both query params; CheckFreshness(now, maxExpires, skew) called in Provider.Verify BEFORE the cache lookup. Config gains MaxTokenExpires (default 15min) + ClockSkew (default 5min). Provider.now func() time.Time is test-injectable. 10 new tests cover happy-path, expired, future-token, exceeds-cap, edge-of-skew.
• M3 419f20f — AllowedTenants []string added with three explicit modes (single-tenant / multi+allowlist / multi+empty="any-tenant globally"). Factory rejects allowed_tenants in single-tenant mode (catches typo). Case-insensitive match. Validator emits warning on the "any-tenant globally" shape so it's loud. --auth-azure-allowed-tenant flag (repeatable). 5 tests. resolveIssuer docstring rewritten to spell out all three modes.
• M4 57d12a5 — refineTokenKind(structural, identity.Source) runs post-verify; gcp_iap → iap_jwt regardless of what Bearer was on the wire. Failure paths still use structural kind (no identity to read). Counter-test confirms oidc source does NOT trigger upgrade.
• M5 ca92921 — KnownAuthProviderSettings closed whitelist per provider type. ValidateAuthConfig emits warning per unknown key. FilterKnownSettings exported as defense-in-depth filter, wired into handlers_create.go BEFORE validation/scaffolding. End-to-end test TestHandleCreateAgent_FiltersUnknownAuthSettings asserts evil_key POST doesn't survive into the scaffold.

NITs — all 7 cleared (745c024)

• gcp_iap classifyJWTErr now uses errors.Is(err, jwt.ErrTokenExpired) etc. with a thoughtful special case for ErrTokenSignatureInvalid (which wraps BOTH alg-confusion and real bad-signature in golang-jwt v5). Substring matches retained only for keyFunc messages WE control.
• gcp_iap JWKS now per-kid merge on success — partial-but-valid rotation responses don't drop in-flight kids.
• azure_ad GraphCache Get/Put defensive copy via append([]string(nil), ...).
• needsYAMLQuoting extended with looksNumeric() — hex (0x), octal (0o), binary (0b), leading-zero (010), scientific (1e10), float (3.14), signed (±N), .inf/.nan in either case.
• identity_cache_test switched to strconv.Itoa(i) — eviction-threshold test now actually reaches 10,001 distinct keys (was collapsing surrogates to U+FFFD before).
• Two req, _ := http.NewRequestWithContext(...) antipatterns fixed (gcp_iap + azure_ad).
• New TestProvider_HS256WithECPublicKeyAsSecret_Rejected pins the most dangerous alg-confusion shape (attacker HMACs with the public key bytes from JWKS).

Gates

• go test -race -count=1 — 9 packages green (auth + all 7 providers + security + validate + cli/cmd + cli/tui/steps + forge-ui)
• golangci-lint — clean across all 3 modules
• gofmt — clean

Verdict

LGTM — ready to merge. Every blocker, major, and nit from the prior review has a focused fix commit with regression tests. The B1-B3 commit's preemptive extension to gcp_iap/iap_jwks.go and the M3 commit's three-mode explicit operational semantic show good adversarial-thinking discipline. Nicely done.