Skip to content

feat(api-proxy): attribute requests to the main agent or a sub-agent via x-awf-caller - #10222

Merged
lpcox merged 4 commits into
mainfrom
copilot/api-proxy-attribute-requests
Oct 18, 2026
Merged

lpcox merged 4 commits into
mainfrom
copilot/api-proxy-attribute-requests

Conversation

Copilot AI commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

AWF records don't say which agent made a request: purpose exists only for router classifier calls. Downstream consumers therefore match AWF token records to engine-reported sub-agent requests by model, token counts and order, which breaks easily. This PR adds an optional x-awf-caller request header. AWF parses it at ingress, always strips it before forwarding upstream, and records it as attribution metadata on token-usage.jsonl and model-routing.jsonl request records.

Header contract

  • Accepted values are main or subagent/<agentName>, at most 128 characters, and the whole value must match ^(?:main|subagent/[A-Za-z0-9_.:/-]+)$. AWF treats any other value as absent.
  • A rejected value is never logged. AWF logs only caller_header_rejected with a fixed reason (too_long / invalid_format / not_a_string).
  • The header is in STRIPPED_HEADERS, so it is never sent upstream, valid or not. That covers buildRequestHeaders, cross-provider fallback and the WebSocket tunnel.

Ingress

  • New pure module caller-header.js (added to the Dockerfile COPY list) with parseCallerHeader, callerHeaderRejection and callerRecordFields.
  • server-factory.js parses the header once into req.awfCaller: in the HTTP handler before routing?.observeRequest, and in the upgrade handler.
  • req.awfCaller is kept separate from the trusted req.awfRequestContext. Steering, guards, budget accounting, the classifier abort signal and response handling still read only awfRequestContext.purpose. The header cannot change behavior.

Recording

callerRecordFields(awfPurpose, caller) is the single source for both record types, so the two always agree for the same request_id:

Request purpose agent_name caller_source
x-awf-caller: main agent — caller_header
x-awf-caller: subagent/<name> subagent <name> caller_header
AWF classifier call (any header ignored) routing_classification — awf
No header / invalid header — — —
  • Token records: the caller is passed through upstream-token.js → trackTokenUsage → buildAndWriteTokenRecord, and through websocket-tunnel.js → token-tracker-ws.js. With no valid caller and no AWF purpose, the record is unchanged from main.
  • Behavior change for existing consumers: classifier rows now also carry caller_source: "awf".
  • Routing request records: requestTelemetry adds the same fields, and agent_name and caller_source are added to FIELDS in routing-persistence.js.

Schema and docs

  • schemas/token-usage.schema.json:
    • The purpose enum is now routing_classification | agent | subagent.
    • New agent_name (pattern ^[A-Za-z0-9_.:/-]+$, maxLength 119).
    • New caller_source (caller_header | awf).
  • docs/api-proxy-sidecar.md has a new "Caller attribution" section, and docs/awf-config-spec.md §13a.1 gains the request-record rules. Both document the trust model: the header is an unauthenticated declaration from inside the sandbox. Consumers should take credits, token counts and served models from AWF's own record and use the caller fields only for attribution.

Acceptance cases → tests

  1. Sub-agent:
    • token-tracker.schema.test.js › buildTokenUsageRecord records a sub-agent caller
    • upstream-token.test.js › passes the parsed x-awf-caller from the request to trackTokenUsage
    • request-headers.test.js › buildRequestHeaders strips x-awf-caller before forwarding upstream
  2. Main agent: token-tracker.schema.test.js › buildTokenUsageRecord records the main-agent caller without agent_name
  3. No header: token-tracker.schema.test.js › buildTokenUsageRecord is unchanged when no caller is present. It uses a fixed clock and a strict deep-equal against main's output.
  4. Invalid value:
    • caller-header.test.js › parseCallerHeader rejects invalid values (table-driven)
    • request-headers.test.js › buildRequestHeaders strips an invalid x-awf-caller
  5. Classifier:
    • token-tracker.schema.test.js › buildTokenUsageRecord prefers the AWF purpose over a caller header
    • routing-provider-executor.test.js › classifier requests carry no x-awf-caller and record caller_source awf
  6. Routing records:
    • routing-observation.test.js › request records carry the same caller fields as the token record
    • routing-persistence.test.js › persists agent_name and caller_source on request records

Additional tests:

  • server-factory.test.js: header parsing for HTTP and upgrade requests, including a check that a rejected value never reaches the log.
  • token-tracker.schema.test.js: records written through the HTTP and WebSocket tracking paths.

Known failure on main

containers/api-proxy/upstream-http.test.js fails to parse on main (Identifier 'EventEmitter' has already been declared). It is unrelated to this change and not fixed here. The latest Build Verification runs on main are also failing; their logs haven't been checked, so this test hasn't been confirmed as the cause.

Copilot AI changed the title [WIP] Add x-awf-caller header for request attribution feat(api-proxy): attribute requests to the main agent or a sub-agent via x-awf-caller Oct 18, 2026
Copilot AI requested a review from SivaKesava1 October 18, 2026 20:57
@lpcox
lpcox marked this pull request as ready for review October 18, 2026 21:07
Copilot AI max review requested due to automatic review settings October 18, 2026 21:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Changes to public purpose semantics and security-sensitive ingress paths warrant maintainer sign-off on downstream compatibility.

1 open issue
What changed in this PR

Adds optional caller attribution to API-proxy records, helping consumers distinguish main-agent and sub-agent requests without fragile usage matching.

Changes:

  • Parses and strips x-awf-caller while keeping attribution separate from trusted request controls.
  • Propagates caller fields through HTTP, WebSocket, token, and routing records.
  • Updates schemas, documentation, and regression tests.
File Description
schemas/​token-usage.schema.json Defines caller attribution fields.
docs/​awf-config-spec.md Specifies recording and trust rules.
docs/​api-proxy-sidecar.md Documents header usage and limitations.
containers/​api-proxy/​websocket-tunnel.js Passes caller metadata to tracking.
containers/​api-proxy/​upstream-token.test.js Tests HTTP caller propagation.
containers/​api-proxy/​upstream-token.js Forwards parsed caller metadata.
containers/​api-proxy/​token-tracker.schema.test.js Tests attribution and compatibility.
containers/​api-proxy/​token-tracker-ws.js Records WebSocket caller metadata.
containers/​api-proxy/​token-tracker-http.js Carries attribution through finalization.
containers/​api-proxy/​token-persistence.js Applies shared attribution fields.
containers/​api-proxy/​server-factory.test.js Tests ingress parsing and safe logging.
containers/​api-proxy/​server-factory.js Parses HTTP and upgrade headers.
containers/​api-proxy/​routing-provider-executor.test.js Tests trusted classifier attribution.
containers/​api-proxy/​routing-persistence.test.js Tests caller-field persistence.
containers/​api-proxy/​routing-persistence.js Allows caller fields in records.
containers/​api-proxy/​routing-observation.test.js Checks token/routing attribution agreement.
containers/​api-proxy/​routing-observation.js Adds attribution to request telemetry.
containers/​api-proxy/​request-headers.test.js Tests caller-header stripping.
containers/​api-proxy/​proxy-utils.js Marks caller header for removal.
containers/​api-proxy/​Dockerfile Includes the attribution module.
containers/​api-proxy/​caller-header.test.js Tests parsing and trusted-purpose precedence.
containers/​api-proxy/​caller-header.js Centralizes validation and attribution mapping.

Review effort: Max

startTime,
metrics,
requestModel: requestModel || extractRequestModelFromUrl(req.url),
caller: req.awfCaller || null,
@lpcox

lpcox commented Oct 18, 2026

Copy link
Copy Markdown
Collaborator

@copilot address review feedback

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot AI commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

@copilot address review feedback

Addressed in 5e081d6. Added a successful CONNECT tunnel regression verifying the tracker receives req.awfCaller and the serialized upgrade strips x-awf-caller using the real header filter. All 168 targeted tests pass; secret scanning is clean. Read-only review found no issues after automated review was unavailable. CI logs confirm main’s unrelated duplicate EventEmitter declaration remains.

Copilot AI requested a review from lpcox October 18, 2026 21:30
@github-actions

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Documentation Preview

Documentation has been built for this PR.

Download preview artifact

To view locally:

  1. Download the docs-preview-pr-10222 artifact from the workflow run
  2. Unzip and open index.html in your browser

Built from commit 658cfad

@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 93.41% 93.42% 📈 +0.01%
Statements 92.00% 92.01% 📈 +0.01%
Functions 91.06% 91.06% ➡️ +0.00%
Branches 85.92% 85.92% ➡️ +0.00%
📁 Per-file Coverage Changes (2 files)
File Lines (Before → After) Statements (Before → After)
src/enclave/host-executor-server.ts 91.0% → 91.0% (+0.00%) 89.7% → 90.0% (+0.29%)
src/log-directory-setup.ts 96.8% → 100.0% (+3.18%) 96.9% → 100.0% (+3.13%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@lpcox

lpcox commented Oct 18, 2026

Copy link
Copy Markdown
Collaborator

Copilot AI commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

@copilot fix the failing ci check https://github.com/github/gh-aw-firewall/actions/runs/39168061206/job/117795554691?pr=10222

Removed the duplicate EventEmitter import in 6443022; the API-proxy suite passes (123 suites, 2,553 tests).

@github-actions

Copy link
Copy Markdown
Contributor

🔑 Smoke Copilot BYOK AOAI (api-key) is testing Azure OpenAI BYOK (api-key) mode on this pull request...

@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

🛡️ Egress verdict from Smoke Copilot Network Isolation

@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

🔮 The oracle has spoken through Smoke Codex

@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

✅ Build Test Suite completed successfully!

Generated by Build Test Suite for #10222

@github-actions

Copy link
Copy Markdown
Contributor

🪪 Smoke Copilot BYOK AOAI (Entra) is testing Azure OpenAI BYOK (Entra / GitHub OIDC) mode on this pull request...

@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

Tested by Smoke Chroot

@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Claude passed

Generated by Smoke Claude for #10222

@github-actions

Copy link
Copy Markdown
Contributor

Cloud Hypervisor + Copilot smoke test

  1. Merged PR lookup: PASS
  2. github.com connectivity: PASS (HTTP 200)
  3. File write/read: PASS
  4. example.com blocked: PASS (HTTP 000)
    Overall: PASS

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

EGRESS_RESULT allow=pass deny=pass

  • ✅ Allowed domain (api.github.com → 200)
  • ✅ Blocked domain (example.com blocked)
    Overall: PASS — @Copilot

🛡️ Egress verdict from Smoke Copilot Network Isolation
Add label ready-for-aw to run again

@github-actions github-actions Bot added the smoke-copilot-network-isolation Copilot network-isolation egress smoke test label Oct 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Claude Engine Validation

Check Status
API ✅ PASS
GitHub CLI ✅ PASS
File access ✅ PASS

Overall result: PASS

Generated by Smoke Claude for #10222 · claude · haiku45 · 49.1 AIC · ⊞ 6.1K · ◷
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Services smoke test: ✅ Redis PONG, ✅ pg_isready, ✅ SELECT 1. Overall: PASS

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke test (Copilot):

  • GitHub MCP ✅ (latest PR: "chore(deps): bump the all-github-actions group with 2 updates")
  • GitHub.com connectivity ✅ (200)
  • File write/read ✅

Overall: PASS
cc @Copilot @SivaKesava1

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Direct) Mode ✅ PASS

  • ✅ GitHub MCP connectivity
  • ✅ GitHub.com connectivity (HTTP 200)
  • ✅ File write/read test
  • ✅ BYOK inference test (agent → api-proxy → api.githubcopilot.com)

Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY) via api-proxy sidecar.

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

OTEL smoke test

  • ✅ S1: otel.js loads (exports startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, ...)
  • ✅ S2: OTEL tests 68/68 passed (3 suites)
  • ✅ S3: env forwarding: trace/parent span IDs in env-passthrough.ts; OTLP vars in api-proxy-env-config.ts
  • ✅ S4: onUsage hook present in token-tracker-http.js
  • ✅ S5: diagnostics: otel.jsonl has 1 record (no export errors observed)

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test

  • feat(api-proxy): record provider error codes on upstream-error and failed request records ✅
  • Share a release-pinned guest base across Cloud Hypervisor VM profiles ✅
  • GitHub reads ✅
  • Playwright title check ❌
  • File write/read ✅
  • Build ✅
  • Overall: FAIL

🔮 The oracle has spoken through Smoke Codex
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Chroot Version Comparison

Runtime Host Version Chroot Version Match?
Python Python 3.12.15 Python 3.12.15 ✅ YES
Node.js v24.21.0 v2.98.0 ❌ NO
Go go1.22.12 go1.22.12 ✅ YES

Result: Not all tests passed (Node.js mismatch), so the smoke-chroot label was not added.

Tested by Smoke Chroot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia ✅ 1/1 passed ✅ PASS
Bun hono ✅ 1/1 passed ✅ PASS
C++ fmt ✅ N/A ✅ PASS
C++ json ✅ N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world ✅ N/A ✅ PASS
.NET json-parse ✅ N/A ✅ PASS
Go color ✅ ok ✅ PASS
Go env ✅ ok ✅ PASS
Go uuid ✅ ok ✅ PASS
Java gson ✅ 1/1 passed ✅ PASS
Java caffeine ✅ 1/1 passed ✅ PASS
Node.js clsx ✅ all passed ✅ PASS
Node.js execa ✅ all passed ✅ PASS
Node.js p-limit ✅ all passed ✅ PASS
Rust fd ✅ 1/1 passed ✅ PASS
Rust zoxide ✅ 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — PASS

Notes: the Bun and Deno install scripts were blocked by the sandbox, so I used the npm bun (1.4.3) and deno (2.9.6) packages. .NET and Maven needed relocated HOME/XDG_RUNTIME_DIR and a temporary local repo path because the default locations were not writable.

Generated by Build Test Suite for #10222 · copilot · auto · 27.7 AIC · ⊞ 12K · ◷
Add label ready-for-aw to run again

@lpcox
lpcox merged commit ac52b79 into main Oct 18, 2026
140 of 146 checks passed
@lpcox
lpcox deleted the copilot/api-proxy-attribute-requests branch October 18, 2026 22:44

This branch had an error being deployed

1 failed deployment
aoai-model — 64430220 Deployed Oct 18, 2026 by lpcox via conclusion #2027
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

api-proxy: attribute requests to the main agent or a sub-agent via an x-awf-caller header

4 participants