fix: update WASM golden tests for v0.0.412/v0.31.0 bump; harden CI Failure Doctor#17139
fix: update WASM golden tests for v0.0.412/v0.31.0 bump; harden CI Failure Doctor#17139
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Updates test fixtures and CI Failure Doctor instructions to accommodate recent version bumps and prevent silent/no-output CI doctor runs.
Changes:
- Regenerates WASM golden fixtures to reflect updated Copilot CLI / GitHub MCP Server (and related) version strings.
- Updates
ci-doctor.mdto fetch log content directly viaget_job_logs(... return_content=true, tail_lines=200)and discourageweb-fetchon blob URLs. - Adds an explicit “Mandatory Output Requirement” section to reduce silent CI doctor completions.
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/basic-claude.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/basic-codex.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/basic-copilot.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/claude-with-network.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/codex-with-tools.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/dispatch-with-inputs.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/issue-trigger.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/pr-trigger.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/push-trigger.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/push-with-head-commit.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/schedule-trigger.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/smoke-claude.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/smoke-codex.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/smoke-copilot.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/smoke-test-tools.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/strict-mode.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/with-expressions.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/with-imports.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/with-permissions.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/with-safe-outputs.golden | Updates golden output version strings for regenerated fixture. |
| pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/with-tools.golden | Updates golden output version strings for regenerated fixture. |
| .github/workflows/ci-doctor.md | Adjusts log-retrieval guidance and adds an explicit safe-output requirement section. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| You **MUST** always end by calling exactly one of these safe output tools before finishing: | ||
|
|
||
| - **`create_issue`**: For actionable CI failures that require developer attention | ||
| - **`add_comment`**: To comment on an existing related issue | ||
| - **`noop`**: When no action is needed (e.g., CI was successful, or failure is already tracked) | ||
| - **`missing_data`**: When you cannot gather the information needed to complete the investigation | ||
|
|
||
| **Never complete without calling a safe output tool.** If in doubt, call `noop` with a brief summary of what you found. |
There was a problem hiding this comment.
The requirement to call "exactly one" safe output tool conflicts with earlier instructions that can require multiple safe-output actions in a single run (e.g., add a comment and then close duplicates via update_issue). Consider changing this to "at least one" (or "end with noop only if no other safe output tool was called") so the workflow can both report findings and perform the necessary follow-up actions without violating the rule.
| You **MUST** always end by calling exactly one of these safe output tools before finishing: | |
| - **`create_issue`**: For actionable CI failures that require developer attention | |
| - **`add_comment`**: To comment on an existing related issue | |
| - **`noop`**: When no action is needed (e.g., CI was successful, or failure is already tracked) | |
| - **`missing_data`**: When you cannot gather the information needed to complete the investigation | |
| **Never complete without calling a safe output tool.** If in doubt, call `noop` with a brief summary of what you found. | |
| You **MUST** always call at least one of these safe output tools before finishing (you may call multiple if needed, but only use `noop` when no other safe output tool is appropriate): | |
| - **`create_issue`**: For actionable CI failures that require developer attention | |
| - **`add_comment`**: To comment on an existing related issue | |
| - **`noop`**: When no action is needed (e.g., CI was successful, or failure is already tracked) | |
| - **`missing_data`**: When you cannot gather the information needed to complete the investigation | |
| **Never complete without calling a safe output tool.** If in doubt, and no other safe output tool applies, call `noop` with a brief summary of what you found. |
The version bump (Copilot 0.0.412, GitHub MCP Server v0.31.0) changed compiled lock file output but stale WASM golden files caused
TestWasmGolden_CompileFixturesto fail across all 21 fixtures. Separately, the CI Failure Doctor ran but silently produced no safe output — the agent downloaded raw blob log URLs viaweb-fetchinstead of inline log content, overwhelming its context window.Changes
WASM golden files (
pkg/workflow/testdata/wasm_golden/): Regenerated all 21 fixtures viamake update-wasm-goldento reflect updated version strings in compiled lock files.ci-doctor.md— Phase 2 log retrieval: Changed instruction from genericget_job_logsto requirereturn_content=true, tail_lines=200, avoiding the pattern of fetching multi-MB blob storage URLs throughweb-fetch:ci-doctor.md— Mandatory output: Added a prominent⚠️ Mandatory Output Requirementsection requiring the agent to always end by calling one ofcreate_issue,add_comment,noop, ormissing_data— preventing silent no-output completions.ci-doctor.lock.yml: Recompiled from updated markdown.Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
https://api.github.com/graphql/usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GO111MODULE 64/bin/go /opt/hostedtoolc-10 -ato�� -bool -buildtags /usr/bin/git -errorsas -ifaceassert -nilfunc git(http block)https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1/usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha runs/20260220-121527-27612/test-3472847007/.github/workflows GO111MODULE g_.a l GOMOD GOMODCACHE o "��� Warning: .github/aw/actions-lock.json does not exist yet"-test.v=true env -json GO111MODULE(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v3/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha -json GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env 1527-27612/test-760875293/.github/workflows GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq .object.sha eutil.test 42623a36614a9d0d87915791:go.mod ortcfg.link GOINSECURE GOMOD GOMODCACHE FpiJ9ENbZatjTay5oJ/BhqTCoRMGewfss9ZXZGY/X4XoDkfiiEtxJ64HjgrP env -json GO111MODULE 0/x64/bin/node GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq .object.sha bRTSsYcqe GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE ortcfg env g/stringutil/ansi.go g/stringutil/identifiers.go ache/go/1.25.0/x64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE ache/go/1.25.0/x64/pkg/tool/linux_amd64/compile(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq .object.sha .actor }}, Unsafe: ${{ secrets.TOKEN }} go /usr/bin/git O8TU/veH-Nz2-QxBgit GO111MODULE 64/bin/go git conf�� user.email test@example.com /usr/bin/git che/go-build/6e/git GOPROXY 64/bin/go git(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v5/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha image:v1.0.0 GO111MODULE 7686675/b405/vet.cfg GOINSECURE GOMOD GOMODCACHE /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet -ato�� -bool -buildtags /usr/bin/git -errorsas -ifaceassert -nilfunc git(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha /tmp/gh-aw-test-runs/20260220-121527-27612/test-1317333712 status /usr/bin/git .github/workflowgit k/gh-aw/gh-aw/pkrev-parse 64/bin/go git rev-�� --show-toplevel /opt/hostedtoolc-test.v=true /opt/hostedtoolcache/node/24.13.0/x64/bin/node b/workflows -trimpath 64/bin/go /opt/hostedtoolcache/node/24.13.0/x64/bin/node(http block)https://api.github.com/repos/actions/github-script/git/ref/tags/v8/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE go env ck 'scripts/**/*GOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -c=4 -nolocalimports -importcfg /tmp/go-build1777686675/b391/importcfg -pack /home/REDACTED/work/gh-aw/gh-aw/pkg/envutil/envutil.go /home/REDACTED/work/gh-aw/gh-aw/pkg/envutil/envutil_test.go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -c=4 -nolocalimports -importcfg /tmp/go-build1777686675/b394/importcfg -pack /home/REDACTED/work/gh-aw/gh-aw/pkg/fileutil/fileutil.go /home/REDACTED/work/gh-aw/gh-aw/pkg/fileutil/fileutil_test.go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/actions/setup-go/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha -json GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/actions/setup-node/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha -json GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env 1527-27612/test-1317333712 GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts/usr/bin/gh gh run download 1 --dir test-logs/run-1 GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts/usr/bin/gh gh run download 12345 --dir test-logs/run-12345 GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts/usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts/usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE x_amd64/link GOINSECURE GOMOD GOMODCACHE x_amd64/link env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE i8/wl1tDiMkVqxwJLLHK96I/vmtrAJLFtest@example.com(http block)https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts/usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts/usr/bin/gh gh run download 4 --dir test-logs/run-4 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go estl�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts/usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/workflows/usr/bin/gh gh workflow list --json name,state,path -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE erignore env -json 42623a36614a9d0dGOMOD 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 b/gh-aw/cmd/gh-a-o GOMODCACHE go env -JoJ/SO7kYnwtxWj-p GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE 7546152/b370/imp-goversion(http block)/usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE x_amd64/link env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE L1/BH39Jih_oyS2LvJ_3yZB/SkE-TZwGBfwFK-8npHOd(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999/usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha ty-test.md GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env 760875293/.github/workflows GO111MODULE x_amd64/link GOINSECURE GOMOD GOMODCACHE x_amd64/link(http block)https://api.github.com/repos/nonexistent/repo/actions/runs/12345/usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet(http block)https://api.github.com/repos/owner/repo/actions/workflows/usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE ache/go/1.25.0/xGO111MODULE env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE ache/go/1.25.0/xGO111MODULE env 7546152/b365/_pkGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/owner/repo/contents/file.md/tmp/go-build1777686675/b380/cli.test /tmp/go-build1777686675/b380/cli.test -test.testlogfile=/tmp/go-build1777686675/b380/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE go ache�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/test-owner/test-repo/actions/secrets/usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name -json GO111MODULE 64/bin/go GOINSECURE GOMOD erignore go env -json GO111MODULE 64/bin/go GOINSECURE 42623a36614a9d0d-c GOMODCACHE go(http block)If you need me to access, download, or install something from one of these locations, you can either:
Original prompt
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.