Skip to content

feat: add pre-steps to agent job for same-job token minting (with import support)#25242

Merged
pelikhan merged 4 commits intomainfrom
copilot/fix-checkout-token-runtime-issue
Apr 8, 2026
Merged

feat: add pre-steps to agent job for same-job token minting (with import support)#25242
pelikhan merged 4 commits intomainfrom
copilot/fix-checkout-token-runtime-issue

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 8, 2026

Token-minting actions (e.g. octo-sts, actions/create-github-app-token) call ::add-mask:: on their output. GitHub Actions runner v2.308+ silently drops masked values crossing job boundaries, so checkout.token: ${{ needs.auth.outputs.token }} arrives empty at runtime even though compilation succeeds.

Solution

New pre-steps frontmatter field: steps that run at the very start of the agent job, before checkout and the subsequent built-in steps. Tokens minted here stay within the same job and are referenceable via ${{ steps.<id>.outputs.token }} without hitting the masked-value drop. Pre-steps defined in imported agentic workflows are also merged in (imported steps run first, then the main workflow's pre-steps).

pre-steps:
  - name: Mint short-lived token
    id: mint
    uses: some-org/token-minting-action@v1
    with:
      scope: target-org/target-repo

checkout:
  - repository: target-org/target-repo
    path: target
    token: ${{ steps.mint.outputs.token }}
    current: false
  - path: .

Changes

  • Schema (main_workflow_schema.json): add pre-steps property
  • Types (frontmatter_types.go, compiler_types.go): PreSteps fields on FrontmatterConfig and WorkflowData
  • Orchestrator (compiler_orchestrator_workflow.go): processAndMergePreSteps with action pinning and import merging, called alongside processAndMergeSteps / processAndMergePostSteps; processAndMergePostSteps updated to also consume MergedPostSteps from imports (pre-existing gap)
  • YAML generation (compiler_yaml.go, compiler_yaml_main_job.go): generatePreSteps emitted after OTLP masking, before checkout app-token minting and checkout steps
  • Validation (strict_mode_steps_validation.go): pre-steps included in secrets strict/non-strict check alongside steps and post-steps
  • Import system (import_field_extractor.go, import_processor.go): pre-steps extracted from imported workflow files and surfaced as ImportsResult.MergedPreSteps for merging

Pre-steps allow users to mint short-lived tokens or perform other setup
that must run in the same job as checkout. This avoids the GitHub Actions
runner masked-value bug (v2.308+) where tokens minted in a separate job
are silently dropped when passed via job outputs.

Users can now reference pre-step outputs in checkout.token:
  pre-steps:
    - name: Mint token
      id: mint
      uses: some-org/token-action@v1
  checkout:
    - repository: target-org/target-repo
      token: ${{ steps.mint.outputs.token }}

Changes:
- JSON schema: add pre-steps property
- FrontmatterConfig: add PreSteps field
- WorkflowData: add PreSteps field
- compiler_orchestrator_workflow: add processAndMergePreSteps
- compiler_yaml: add generatePreSteps function
- compiler_yaml_main_job: emit pre-steps before checkout/app-token steps
- strict_mode_steps_validation: include pre-steps in secret validation
- compiler_presteps_test: add tests for pre-steps functionality

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/86c17acc-cd92-4ba9-9c18-c2f101070c71

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix checkout.token runtime issue with masked output feat: add pre-steps to agent job for same-job token minting Apr 8, 2026
Copilot AI requested a review from pelikhan April 8, 2026 05:29
@pelikhan pelikhan marked this pull request as ready for review April 8, 2026 05:30
Copilot AI review requested due to automatic review settings April 8, 2026 05:30
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for a new pre-steps frontmatter section so workflows can run user-defined steps at the start of the agent job (before checkout and other built-in steps), enabling same-job token minting that avoids masked job-output drops in newer GitHub Actions runners.

Changes:

  • Extend schema and Go frontmatter/compiler types to support pre-steps.
  • Parse/merge pre-steps in the orchestrator (including action pinning) and emit them early in the agent job YAML.
  • Update strict-mode secrets validation and add compilation tests covering pre-steps ordering and checkout-token usage.
Show a summary per file
File Description
pkg/workflow/strict_mode_steps_validation.go Includes pre-steps in secrets-expression validation.
pkg/workflow/frontmatter_types.go Adds typed PreSteps to frontmatter config + map conversion.
pkg/workflow/compiler_yaml.go Implements generatePreSteps YAML emission helper.
pkg/workflow/compiler_yaml_main_job.go Emits pre-steps early in agent job step sequence.
pkg/workflow/compiler_types.go Adds PreSteps field to WorkflowData.
pkg/workflow/compiler_presteps_test.go Adds tests validating pre-steps ordering and checkout token reference.
pkg/workflow/compiler_orchestrator_workflow.go Extracts and action-pins pre-steps from frontmatter.
pkg/parser/schemas/main_workflow_schema.json Adds pre-steps to workflow frontmatter schema.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 8/8 changed files
  • Comments generated: 2

Comment on lines +23 to +28
// Add pre-steps before checkout and any other built-in steps.
// This allows users to mint short-lived tokens (via custom actions) in the same
// job as checkout, so the tokens are never dropped by the GitHub Actions runner's
// add-mask behaviour that silently suppresses masked values across job boundaries.
// Step outputs are available as ${{ steps.<id>.outputs.<name> }} and can be
// referenced directly in checkout.token.
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment says pre-steps run "before checkout and any other built-in steps", but in this function OTLP header masking is emitted before pre-steps, and the overall agent job can also prepend setup steps earlier (see buildMainJob in pkg/workflow/compiler_main_job.go). Suggest rewording to clarify that pre-steps run before checkout and the subsequent built-in steps in the agent job (but not necessarily before every earlier compiler-injected setup step).

Suggested change
// Add pre-steps before checkout and any other built-in steps.
// This allows users to mint short-lived tokens (via custom actions) in the same
// job as checkout, so the tokens are never dropped by the GitHub Actions runner's
// add-mask behaviour that silently suppresses masked values across job boundaries.
// Step outputs are available as ${{ steps.<id>.outputs.<name> }} and can be
// referenced directly in checkout.token.
// Add pre-steps before checkout and the subsequent built-in steps in this agent job.
// This allows users to mint short-lived tokens (via custom actions) in the same
// job as checkout, so the tokens are never dropped by the GitHub Actions runner's
// add-mask behaviour that silently suppresses masked values across job boundaries.
// Step outputs are available as ${{ steps.<id>.outputs.<name> }} and can be
// referenced directly in checkout.token. Some compiler-injected setup steps may
// still be emitted earlier than these pre-steps.

Copilot uses AI. Check for mistakes.
Comment on lines +28 to +36
pre-steps:
- name: Mint short-lived token
id: mint
uses: some-org/token-minting-action@v1
with:
scope: target-org/target-repo
steps:
- name: Custom Setup Step
run: echo "Custom setup"
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test YAML uses unpinned action tags (e.g. some-org/token-minting-action@v1 / some-org/token-action@v1). Because action pinning attempts dynamic resolution via gh api on cache misses (see pkg/workflow/action_pins.go:GetActionPinWithData), this makes the unit test non-hermetic and can trigger network calls / firewall failures. Consider switching these examples to pinned SHAs (like other tests do) or seeding the ActionCache / disabling the ActionResolver for this test so it stays offline/deterministic.

Copilot uses AI. Check for mistakes.
@pelikhan
Copy link
Copy Markdown
Collaborator

pelikhan commented Apr 8, 2026

@copilot review comments, make it work for imported agentic workflows

Copilot AI and others added 2 commits April 8, 2026 05:53
…comments

- Add preStepsBuilder to importAccumulator in import_field_extractor.go
  and extract pre-steps from imported workflow files
- Add MergedPreSteps field to ImportsResult in import_processor.go
- Update processAndMergePreSteps to accept importsResult and merge
  imported pre-steps before the main workflow's pre-steps
- Update processAndMergePostSteps to accept importsResult and merge
  imported post-steps after the main workflow's post-steps (fixing a
  pre-existing gap where MergedPostSteps was built but never consumed)
- Fix comment wording in compiler_yaml_main_job.go per reviewer suggestion
- Use pinned SHAs (40-char) in test fixtures to avoid network calls
- Add unit tests for processAndMergePreSteps and imported post-steps

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/088642a3-c500-49a7-a249-a7db5353583e

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…rror in tests

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/088642a3-c500-49a7-a249-a7db5353583e

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 8, 2026

@copilot review comments, make it work for imported agentic workflows

Done in commit a511f60 (with a follow-up fix in 0668ade).

Pre-steps now work for imported agentic workflows:

  • Added preStepsBuilder to importAccumulator in import_field_extractor.go — extracts pre-steps from each imported workflow file
  • Added MergedPreSteps field to ImportsResult in import_processor.go
  • Updated processAndMergePreSteps to merge imported pre-steps before the main workflow's pre-steps (same ordering convention as steps)
  • Also fixed processAndMergePostSteps to consume MergedPostSteps from imports (pre-existing gap where the field was built but never used)

Also addressed the reviewer comments:

  • Rewrote the comment in compiler_yaml_main_job.go to accurately reflect that OTLP masking runs before pre-steps
  • Replaced unpinned action tags (@v1) in tests with 40-char SHAs to avoid network calls during unit tests

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE node (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE sh (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw %H %ct %D (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha --show-toplevel go /usr/bin/git on.lock.yml GO111MODULE $name) { has--show-toplevel git rev-�� --show-toplevel go /usr/bin/git -json GO111MODULE ache/go/1.25.8/x--show-toplevel git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha xterm-color sh /usr/bin/gh npx prettier --cgit GOPROXY $name) { has--show-toplevel gh api k/gh-aw/gh-aw/.github/workflows --jq (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha -json GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel go /usr/bin/git .lock.yml GO111MODULE ache/go/1.25.8/x--show-toplevel git rev-�� --show-toplevel go /usr/bin/git -json GO111MODULE /opt/hostedtoolc--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel git /usr/bin/git --show-toplevel sh er: String!, $na--show-toplevel git rev-�� --show-toplevel /usr/bin/gh /usr/bin/git graphql -f (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel go /usr/bin/git -json GO111MODULE 64/bin/go git -C /home/REDACTED/work/gh-aw/gh-aw/.github/workflows rev-parse /usr/bin/git -json GO111MODULE 64/bin/go git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel go /opt/hostedtoolcache/uv/0.11.4/x86_64/bash -json GO111MODULE repository(owne--show-toplevel bash --no�� --noprofile go /usr/bin/git -json GO111MODULE 64/bin/go git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha tags/v4 git /usr/bin/git ithub/workflows config /usr/bin/gh git rev-�� --show-toplevel /usr/bin/gh /usr/bin/git k/gh-aw/gh-aw/.ggit -f me: String!) { --show-toplevel git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha --show-toplevel go /usr/bin/git -json GO111MODULE 64/bin/go git -C /home/REDACTED/work/gh-aw/gh-aw/.github/workflows rev-parse /usr/bin/git -json GO111MODULE 64/bin/go git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha HEAD:.github/workflows/smoke-workflow-call.lock.yml go /usr/bin/infocmp -json GO111MODULE r: $owner, name:--show-toplevel infocmp -1 xterm-color go /usr/bin/git ock.yml GO111MODULE 64/bin/go git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha xterm-color go /opt/hostedtoolcache/go/1.25.8/x64/bin/bash -json GO111MODULE r: $owner, name:--show-toplevel bash --no�� --noprofile go /usr/bin/git s.lock.yml GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha /home/REDACTED/work/gh-aw/gh-aw/.github/workflows rev-parse /usr/bin/gh -json GO111MODULE 64/bin/go /usr/bin/gh api graphql -f /usr/bin/git -f owner=github -f git (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha graphql -f /usr/bin/git -f owner=github -f git rev-�� --show-toplevel go /usr/bin/git -json GO111MODULE repository(owne--show-toplevel git (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq .object.sha ithub/workflows config er: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabl--show-toplevel remote.origin.urgit GOWORK 64/bin/go infocmp -1 xterm-color erena-mcp-server:latest 64/pkg/tool/linux_amd64/compile -json GO111MODULE $name) { has--show-toplevel 64/pkg/tool/linux_amd64/compile (http block)
  • https://api.github.com/repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq .object.sha MFpD/H8VUAZrsjt5GOINSECURE GO111MODULE $name) { hasDiscussionsEnabled } } GOINSECURE GOMOD GOMODCACHE 898935/b400/impoGOPROXY -c k/gh-aw/gh-aw/pkGOSUMDB k/gh-aw/gh-aw/pkGOWORK 64/bin/go GOSUMDB GOWORK 64/bin/go /opt/hostedtoolcache/go/1.25.8/xGO111MODULE (http block)
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq .object.sha vxy_/_JYY_vYdU1VGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE 898935/b410/impoGOPROXY -c k/gh-aw/gh-aw/pkGOSUMDB k/gh-aw/gh-aw/pkGOWORK 64/bin/go GOSUMDB GOWORK 64/bin/go /opt/hostedtoolcconfig (http block)
  • https://api.github.com/repos/github/gh-aw
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw --jq .visibility (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq .object.sha --show-toplevel go /usr/bin/git -json GO111MODULE 64/bin/go git -C /home/REDACTED/work/gh-aw/gh-aw/.github/workflows rev-parse /usr/bin/git -json GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq .object.sha --show-toplevel GOPROXY me: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } GOSUMDB GOWORK 64/bin/go /usr/bin/gh api ithub/workflows -f er: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabl--show-toplevel -f owner=github -f git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq .object.sha kflows/delight.lock.yml rev-parse /usr/bin/git GOSUMDB GOWORK 64/bin/go git rev-�� --show-toplevel go /usr/bin/gh -json GO111MODULE 64/bin/go /usr/bin/gh (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq .object.sha --show-toplevel go /usr/bin/git -json GO111MODULE me: String!) { --show-toplevel git rev-�� --show-toplevel go /usr/bin/git -json l /opt/hostedtoolc--show-toplevel git (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha -json GO111MODULE 64/pkg/tool/linux_amd64/asm GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/asm env -json GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/githubnext/agentics/git/ref/tags/-
    • Triggering command: /usr/bin/gh gh api /repos/githubnext/agentics/git/ref/tags/- --jq .object.sha --check scripts/**/*.js 64/bin/go -d **/*.cjs 64/bin/go go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha -json GO111MODULE $name) { hasDiscussionsEnabled } } GOINSECURE GOMOD GOMODCACHE go env tor.lock.yml GO111MODULE ache/go/1.25.8/x-f GOINSECURE GOMOD GOMODCACHE go (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title feat: add pre-steps to agent job for same-job token minting feat: add pre-steps to agent job for same-job token minting (with import support) Apr 8, 2026
@pelikhan pelikhan merged commit 2204e0f into main Apr 8, 2026
@pelikhan pelikhan deleted the copilot/fix-checkout-token-runtime-issue branch April 8, 2026 05:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

checkout.token from custom jobs: fails at runtime when token-minting action masks output

3 participants