Skip to content

fix(steps): expose shell and atmos outputs to later steps - #2785

Merged
Andriy Knysh (aknysh) merged 6 commits into
cloudposse:mainfrom
sgtoj:fix/step-shell-output-capture
Jul 23, 2026
Merged

Andriy Knysh (aknysh) merged 6 commits into
cloudposse:mainfrom
sgtoj:fix/step-shell-output-capture

Conversation

@sgtoj

@sgtoj Brian Ojeda (sgtoj) commented Jul 22, 2026 •

Copy link
Copy Markdown
Contributor

what

  • Capture successful output from named shell and atmos steps in custom
    commands and workflows.
  • Expose trimmed .value, masked stdout/stderr/exit-code metadata, and declared
    outputs through .steps.<name>.
  • Evaluate declared outputs once after successful command execution, outside the
    retry envelope.
  • Support host execution, persistent workflow containers, and per-step container
    overrides.
  • Preserve legacy runners, retries, live streaming, custom-command shell and
    atmos output: none, terminal sessions, process cleanup, and failure
    propagation.
  • Document command-result behavior and correct the interactive-session capture
    guidance.
  • Add regression coverage for markdown consumers, retries, masking, suppression,
    terminal sessions, output-template failures, and container execution.

why

  • Legacy shell and atmos execution bypassed StepExecutor result storage.
  • Later steps therefore failed with map has no entry for key "<name>", even
    after the producer completed successfully.
  • Retaining the legacy execution paths avoids known Windows regressions and
    child-process cleanup issues while satisfying the documented step-output
    contract.
  • Captured values are masked before entering template context so later rendering
    cannot expose registered secrets.
  • Keeping output evaluation outside retries prevents a template error from
    repeating a successful, potentially non-idempotent command.

validation

  • Audited against the step-output and shell/atmos command-step PRDs.
  • Patch-scoped coverage reported no uncovered added behavior.
  • Passed complete tests for ./cmd, ./internal/exec, ./pkg/runner/step, and
    ./pkg/workflow.
  • Passed focused race tests with shuffled ordering.
  • Verified masking and output: none behavior for custom-command shell and
    atmos steps.
  • Passed go build ./....
  • Passed patch-scoped lint against upstream/main.
  • Built the Docusaurus website successfully.
  • Re-ran the original custom-command and workflow reproductions successfully.
  • Full-repository test wrappers reached and passed every touched package;
    unrelated AWS/XDG assertions and an acceptance-test timeout caused the overall
    local run to remain nonzero.

references

Summary by CodeRabbit

  • New Features
    • Step output capture for shell and atmos is now consistent across local and container execution, including retries and named declared outputs (value, stdout, stderr, exit code).
    • Secret masking is applied to both live output and stored step metadata.
  • Bug Fixes
    • output: none now suppresses streamed output while preserving stored results for downstream steps.
    • Output-template evaluation failures no longer cause retries and prevent storing a step result.
    • Retry behavior now ensures only the final successful attempt’s stored outputs are used.
    • Interactive/TTY sessions still attach directly to the terminal and remain non-capturable.
  • Documentation
    • Updated workflow step output documentation and added a fix note for command output availability.

@sgtoj
Brian Ojeda (sgtoj) requested a review from a team as a code owner July 22, 2026 19:23
@atmos-pro

atmos-pro Bot commented Jul 22, 2026 •

Copy link
Copy Markdown
Contributor

Tip

Atmos Pro  

No affected stacks workflow was detected for this pull request.
If this is expected, no action is needed.
Learn More. Ask AI.

@github-actions github-actions Bot added the size/l Large size PR label Jul 22, 2026
@sgtoj

Copy link
Copy Markdown
Contributor Author

Maintainer action requested: please approve the first-time-contributor workflow runs and add the patch semver label. I attempted to add the label directly, but fork contributors do not have label-management permission.

@coderabbitai

coderabbitai Bot commented Jul 22, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Changes

Shell and atmos steps now capture stdout and stderr, store trimmed command results with metadata and declared outputs, and expose them to later steps. Local, container, workflow, and custom-command paths include coverage for masking, retries, output suppression, and terminal sessions.

Step output capture

Layer / File(s) Summary
Command result storage
pkg/runner/step/command_result.go, pkg/runner/step/command_result_test.go
Adds shared command capture, masking, metadata construction, declared-output evaluation, and persistence.
Container output forwarding
pkg/workflow/container.go, pkg/workflow/*_test.go
Adds capture writers for container execution and verifies terminal sessions remain uncaptured.
Workflow capture wiring
internal/exec/workflow_utils.go, internal/exec/workflow_step_output_capture_test.go, internal/exec/workflow_utils_test.go
Routes workflow shell and atmos execution through result storage and tests local, container, markdown, retry, dry-run, and failure behavior.
Custom command capture
cmd/cmd_utils.go, cmd/custom_command_*, cmd/testing_main_test.go
Captures custom-command shell and atmos output while preserving masking and output: none behavior, with subprocess and retry coverage.
Output contract documentation
docs/fixes/..., website/docs/workflows/..., website/docs/cli/...
Documents trimmed values, stdout/stderr metadata, declared outputs, and non-capturable interactive or TTY sessions.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CommandStep
  participant runCommandStep
  participant ShellOrAtmos
  participant StepResultStore
  participant LaterStep
  CommandStep->>runCommandStep: execute named step
  runCommandStep->>ShellOrAtmos: provide capture writers
  ShellOrAtmos-->>runCommandStep: stdout and stderr
  runCommandStep->>StepResultStore: store value and metadata
  LaterStep->>StepResultStore: resolve .steps.<name>
Loading

Possibly related PRs

Suggested reviewers: aknysh

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes capture and store shell/atmos results for custom commands and workflows, including metadata, retries, masking, and terminal-session caveats.
Out of Scope Changes check ✅ Passed No unrelated code changes stand out; the docs, tests, and container-capture updates all support the output-capture fix.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: exposing shell and atmos outputs to later steps.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/runner/step/command_result.go`:
- Around line 13-33: Update ExecuteAndStoreCommandResult and its callers in
workflow_utils.go to distinguish errors returned by run from failures in
vars.SetWithOutputs/output-template evaluation, using a sentinel or wrapped
error type as appropriate. Ensure retry handling re-executes the command only
for command failures and does not invoke run again after a successful command
whose result post-processing fails.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 8cd09c37-8f74-4622-be46-921701714d6e

📥 Commits

Reviewing files that changed from the base of the PR and between a8bc6e9 and 4e2972a.

📒 Files selected for processing (15)
  • cmd/cmd_utils.go
  • cmd/custom_command_integration_test.go
  • cmd/custom_command_step_output_test.go
  • cmd/testing_main_test.go
  • docs/fixes/2026-07-22-shell-atmos-step-output-capture.md
  • internal/exec/workflow_step_output_capture_test.go
  • internal/exec/workflow_utils.go
  • internal/exec/workflow_utils_test.go
  • pkg/runner/step/command_result.go
  • pkg/runner/step/command_result_test.go
  • pkg/workflow/container.go
  • pkg/workflow/container_helpers_test.go
  • pkg/workflow/container_test.go
  • website/docs/cli/configuration/commands/command/steps.mdx
  • website/docs/workflows/workflows/workflow/steps/outputs.mdx

Comment thread pkg/runner/step/command_result.go Outdated
@codecov

codecov Bot commented Jul 22, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.59155% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.73%. Comparing base (d9196d7) to head (d1b5a50).

Files with missing lines Patch % Lines
cmd/cmd_utils.go 95.65% 1 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2785      +/-   ##
==========================================
+ Coverage   81.69%   81.73%   +0.04%     
==========================================
  Files        1778     1779       +1     
  Lines      170743   170835      +92     
==========================================
+ Hits       139481   139638     +157     
+ Misses      23528    23467      -61     
+ Partials     7734     7730       -4     
Flag Coverage Δ
unittests 81.73% <98.59%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
internal/exec/workflow_utils.go 77.26% <100.00%> (+1.88%) ⬆️
pkg/runner/step/command_result.go 100.00% <100.00%> (ø)
pkg/workflow/container.go 91.44% <100.00%> (+4.47%) ⬆️
cmd/cmd_utils.go 71.23% <95.65%> (+1.55%) ⬆️

... and 11 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@aknysh Andriy Knysh (aknysh) added the patch A minor, backward compatible change label Jul 22, 2026
Evaluate declared outputs once after successful shell and atmos execution so template errors cannot rerun completed commands.
@sgtoj

Copy link
Copy Markdown
Contributor Author

CodeRabbit (@coderabbitai) review

@coderabbitai

coderabbitai Bot commented Jul 22, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@sgtoj

Copy link
Copy Markdown
Contributor Author

Maintainer action requested: please approve the workflow runs for current head 58f9ca885; GitHub marked them action_required after updating this fork branch from main. The required patch label is now present.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
cmd/custom_command_step_output_test.go (1)

115-229: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Masking/suppression tests only cover shell, unlike the sibling tests in this file.

TestCustomCommandShellMasksLiveAndStoredOutput and TestCustomCommandShellOutputNoneCapturesWithoutStreaming hardcode Type: "shell", while the legacy-output test above and the retry test below both loop over []string{"shell", "atmos"}. Given the PR objective to preserve masked live streaming and suppression across both step types, extending these two tests to the same loop would close a coverage gap for the atmos path with minimal new code (the pattern already exists in this file).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cmd/custom_command_step_output_test.go` around lines 115 - 229, Extend
TestCustomCommandShellMasksLiveAndStoredOutput and
TestCustomCommandShellOutputNoneCapturesWithoutStreaming to iterate over both
step types, []string{"shell", "atmos"}, following the existing loop pattern in
the neighboring tests. Apply the selected type to each relevant task and keep
the current masking, suppression, capture, and assertions unchanged for both
paths.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/runner/step/command_result_test.go`:
- Around line 18-20: In the subtest containing ApplyMaskingConfig and
Masker().RegisterValue, register a t.Cleanup callback that uses the package’s
established masking reset/restore helper to restore the prior global iolib
state. Keep the existing masking setup and secret registration unchanged.

---

Nitpick comments:
In `@cmd/custom_command_step_output_test.go`:
- Around line 115-229: Extend TestCustomCommandShellMasksLiveAndStoredOutput and
TestCustomCommandShellOutputNoneCapturesWithoutStreaming to iterate over both
step types, []string{"shell", "atmos"}, following the existing loop pattern in
the neighboring tests. Apply the selected type to each relevant task and keep
the current masking, suppression, capture, and assertions unchanged for both
paths.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: a1a28fdd-2d56-4065-b403-f2f324663734

📥 Commits

Reviewing files that changed from the base of the PR and between 4e2972a and 58f9ca8.

📒 Files selected for processing (9)
  • cmd/cmd_utils.go
  • cmd/custom_command_integration_test.go
  • cmd/custom_command_step_output_test.go
  • docs/fixes/2026-07-22-shell-atmos-step-output-capture.md
  • internal/exec/workflow_step_output_capture_test.go
  • internal/exec/workflow_utils.go
  • pkg/runner/step/command_result.go
  • pkg/runner/step/command_result_test.go
  • website/docs/workflows/workflows/workflow/steps/outputs.mdx
🚧 Files skipped from review as they are similar to previous changes (6)
  • docs/fixes/2026-07-22-shell-atmos-step-output-capture.md
  • website/docs/workflows/workflows/workflow/steps/outputs.mdx
  • cmd/cmd_utils.go
  • internal/exec/workflow_step_output_capture_test.go
  • cmd/custom_command_integration_test.go
  • internal/exec/workflow_utils.go

Comment thread pkg/runner/step/command_result_test.go
Suppress live custom-command atmos streams while retaining captured results, extend masking and suppression coverage to both command step types, and reset masking state after tests.
@sgtoj

Copy link
Copy Markdown
Contributor Author

CodeRabbit (@coderabbitai) review

@sgtoj

Copy link
Copy Markdown
Contributor Author

Maintainer action requested: please approve the workflow runs for current head b11939ba8. The CodeRabbit findings are addressed and both review threads are resolved.

@coderabbitai

coderabbitai Bot commented Jul 22, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cmd/custom_command_step_output_test.go`:
- Around line 236-244: Strengthen the assertions around captureStdoutStderr in
the customCmd.Run test to verify both "produced" and "warning" are absent from
both stdout and stderr. Keep the existing result-file assertion unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 223e840f-d8b9-450a-97a9-be1c4f8e0dc3

📥 Commits

Reviewing files that changed from the base of the PR and between 58f9ca8 and b11939b.

📒 Files selected for processing (4)
  • cmd/cmd_utils.go
  • cmd/custom_command_step_output_test.go
  • docs/fixes/2026-07-22-shell-atmos-step-output-capture.md
  • pkg/runner/step/command_result_test.go
🚧 Files skipped from review as they are similar to previous changes (3)
  • docs/fixes/2026-07-22-shell-atmos-step-output-capture.md
  • pkg/runner/step/command_result_test.go
  • cmd/cmd_utils.go

Comment thread cmd/custom_command_step_output_test.go
@sgtoj

Copy link
Copy Markdown
Contributor Author

Maintainer action requested: please approve workflow runs for current head d1b5a50dd. All CodeRabbit threads are resolved and confirmed addressed.

@sgtoj

Copy link
Copy Markdown
Contributor Author

CodeRabbit (@coderabbitai) review

@coderabbitai

coderabbitai Bot commented Jul 22, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@aknysh
Andriy Knysh (aknysh) merged commit 6426ad2 into cloudposse:main Jul 23, 2026
87 checks passed
@atmos-pro

atmos-pro Bot commented Jul 23, 2026 •

Copy link
Copy Markdown
Contributor

Tip

Atmos Pro  

No affected stacks workflow was detected for this pull request.
If this is expected, no action is needed.
Learn More. Ask AI.

@github-actions

Copy link
Copy Markdown

These changes were released in v1.223.1-rc.4.

This branch was previously deployed

1 inactive deployment
screengrabs — d1b5a50d Deployed Jul 22, 2026 by sgtoj via build #545
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

patch A minor, backward compatible change size/l Large size PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Custom command & workflow shell/atmos steps don't populate .steps.<name> (documented step outputs contract broken)

2 participants