Routed smokes: cover Claude main models on the Copilot and pi engines, the SDK reverse direction, and non-economy modes #68068

Problem

Main has four routed smokes. Each checks one happy path, and all four use mode: economy with goal: cost.

The following paths have no live coverage:

• Copilot CLI on a Claude main model. In economy the router picks a GPT model. Run 38173405739 picked gpt-5.6-luna at effort max, with 4 requests on /responses. The smoke also sets no main endpoint, so M1 isn't checked. On main, the harness derives the Copilot wire API from the selected endpoint and checks it against AWF /reflect (Select Copilot wire APIs from AWF metadata and surface model mismatches #67487), so a Claude main model should send /chat/completions. No CI run checks that.
• Copilot SDK in the reverse direction, with a Claude main model and a GPT sub-agent. Fix Copilot SDK model routing and cross-family sub-agents #67509 made this work, but only the forward direction is a smoke.
• pi on a Claude main model (/v1/messages). Only the golden fixture pi-claude-two-subagents covers it. The live smoke's pool is gpt-5.6-luna only.
• Non-economy modes. mode: auto (the classifier recommends the profile), balanced and robust never run in a smoke. No smoke runs a Claude main model at a high effort (xhigh, max), and none runs a Claude main model that has no effort (claude-haiku-4.5).

Evidence that these paths work today

These runs are in the private sandbox githubnext/gh-aw-routing-sandbox. I audited them with gh aw audit built at v0.91.8 and replayed main's checkModelRoutingEvidence from smoke_model_routing_assertions.cjs against the downloaded evidence.

• Copilot CLI routed to Claude Opus (full 7-model pool, mode: auto):
  • Run 37402330643: auto resolved to robust, claude-opus-5 at effort max, 95 main requests.
  • Run 37505153056: auto resolved to balanced, claude-opus-5 at xhigh, 101 main requests.
  • All main traffic went to /chat/completions with status 200.
  • Both runs predate Select Copilot wire APIs from AWF metadata and surface model mismatches #67487. The router's selection named /v1/messages, so model-routing.jsonl marks each request as deviated on the endpoint. R3 and M1 check the effective request endpoint, so this deviation doesn't fail them.
• pi routed to Claude: run 38181837342 (gh-aw v0.91.8 setup action, pi 1.1.0).
  • Auto resolved to balanced, claude-opus-5 at effort max.
  • The router's selection named /chat/completions, but all 28 effective requests went to /v1/messages, all 200.
  • With mainEndpoint: "/v1/messages", the replay passes E1, E2, R1–R4 and M1.
  • Replay caveat: R1 needs the runner-written aw_info.json (agent/aw_info.json in the audit download). The root aw_info.json in the download has no routing status, so R1 fails on it. Live runs read /tmp/gh-aw/aw_info.json and aren't affected.
• Copilot SDK, forward direction (GPT main, Claude sub-agent):
  • Runs 38078597395 (failed) and 38084069509 (succeeded) come from a custom-driver workflow.
  • Routed run 38090503174: auto resolved to economy, gpt-5.6-luna on /responses, claude-haiku-4.5 sub-agent on /chat/completions.
• Copilot SDK, reverse direction (Claude main, GPT sub-agent):
  • Run 38090505292 (success), plus 38088108078 and 38085680227.
  • Main model claude-sonnet-5 on /chat/completions (3 requests). Sub-agent file-summarizer on gpt-5.4-mini via /responses (2 requests).
  • These runs pinned claude-sonnet-5. No classifier ran and there is no model-routing.jsonl, so no run has yet shown a routed SDK run on a Claude main model.

Proposed fix

Add three new routed smokes as separate workflows, not variants of the existing ones, and use them to cover the non-economy modes. Separate workflows are better because:

• Each smoke needs a pool from a single model family. That keeps the expected main endpoint deterministic whatever the router picks.
• Widening an existing pool would make the endpoint depend on the router's choice.
• The existing smokes keep their current signal.
• smoke-copilot-routed is bound to its current pool by pkg/workflowcontract/routed_smoke_detection_contract_test.go.

Reuse smoke_model_routing_assertions.cjs and the existing smoke pattern: the same triggers (smoke-routing label command, schedule, workflow_dispatch), a 10-minute timeout, and checks printed in the step summary. Keep each workflow cheap: a trivial task, at most a few model requests, and a tool-call cap like the one smoke-pi-routed uses. The names below are suggestions.

1. Copilot CLI routed to Claude (for example smoke-copilot-routed-claude).
   • Use a Claude-only pool, for example claude-sonnet-5. Optionally add claude-haiku-4.5 so the router can also pick a Claude model that has no effort.
   • Check E1, E2, R1–R4, and M1 with main endpoint /chat/completions.
2. pi routed to Claude (for example smoke-pi-routed-claude).
   • Use a Claude-only pool, for example claude-sonnet-5, with main endpoint /v1/messages.
   • The recorded path has a ?beta=true query. The assertion's endpoint normalization already strips it.
   • Before adding claude-haiku-4.5 to this pool, confirm that pi sends no effort for it. Claude Code does send one, which is why smoke-claude-routed excludes haiku.
   • If you add sub-agents, keep their models outside the pool, as C1 requires.
3. Copilot SDK in the reverse direction (for example smoke-copilot-sdk-routed-reverse).
   • Use a Claude-only pool, for example claude-sonnet-5, with main endpoint /chat/completions.
   • Declare one GPT sub-agent outside the pool, for example gpt-5.4-mini on /responses.
   • Check C1 and S1–S4 for that sub-agent, and require declared agent names as smoke-copilot-sdk-routed does.
4. Non-economy modes.
   • Put one new smoke on mode: auto. The Copilot CLI Claude smoke is a good fit, since the sandbox shows auto picking balanced or robust with xhigh or max on Claude there.
   • Put one other new smoke on a higher fixed profile, balanced or robust. pi is a good fit.
   • Leave the four existing smokes on economy.
   • The checks must not depend on which profile or effort the router picks, and they don't need to:
     • R1: the routing status is "selected" and the selected model is in the pool.
     • R2: exactly one classifier request.
     • R3: main traffic used the selected model and got a 200 on an endpoint that model supports.
     • R4: only pool models were used.
     • M1: the endpoint is fixed by the single-family pool.
   • Write the resolved mode, model and effort to the step summary for information, but don't assert them.
   • Higher profiles can pick claude-opus-5 at xhigh or max, so keep the task trivial and cap requests.

Threat detection for each new smoke: either use the detection checks (detectionMain) and meet the contract test (the detection allowed models must equal the agent's compiled allowed models, and the detection model must equal the compiled detection COPILOT_MODEL), or turn detection off as the other three routed smokes do.

Acceptance cases

1. Each new smoke passes on main. Dispatched on main after merge, each new smoke is green and prints PASS for every check. That includes M1 with the endpoint that matches its main model's family:
   • /chat/completions for the Copilot CLI and SDK Claude smokes.
   • /v1/messages for the pi Claude smoke.
   • For the SDK reverse smoke, S3 checks /responses for the GPT sub-agent.
2. Negative unit cases in smoke_model_routing_assertions.test.cjs, built from real record shapes of the sandbox runs above. Redact them and don't invent fields.
   • A Copilot CLI run whose Claude main traffic went to /responses fails M1. Base it on the records of 37402330643 or 37505153056 and change only the request path.
   • An SDK reverse-direction run whose GPT sub-agent went to /chat/completions fails S3. Base it on the records of 38090505292 and change only the sub-agent's request path. That run pinned its model, so it has no classifier or routing records. Supply that evidence the way the existing fixtures in the test file do, and say so in the test.
   • Add the matching positive case for each: the unchanged shapes pass M1 or S3.
   • I checked both changes against main's checker on the audited data. They fail M1 and S3 respectively.
3. Verification plan in the PR. New workflows can't run on their own PR, because activation reads the prompt from the base branch. The PR must say that a maintainer will verify them on main after merge, or in the sandbox before merge, and should link those runs once they exist.

Scope

• Keep the PR to this one fix. Do not change files or behavior unrelated to it.
• Before fixing a failing check, compare it with the same check on main. If it also fails on main, do not fix it in this PR; merge main once main is fixed, and note in the PR which checks are known failures on main.
• This applies even if a bot comment asks to fix a failing check: fix only failures caused by this PR's changes.