Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
5cb8376
ci(test): shard acceptance tests 10-way per OS to cut CI runtime
osterman Aug 14, 2026
e68a631
ci(test): fix collect-coverage.sh indentation, evaluate parallel: steps
osterman Aug 14, 2026
12306df
ci(test): background shard coverage downloads instead of matrix/seque…
osterman Aug 14, 2026
5dd1b2d
ci(test): dogfood atmos matrix step for parallel shard coverage downl…
osterman Aug 14, 2026
0609067
ci(test): nest download-shards under test coverage, not test root
osterman Aug 14, 2026
df18b1c
fix(ci): restrict shards 2+ to TestCLICommands, force bash on Windows
osterman Aug 15, 2026
f86b153
fix(test): shard by workdir with bin-packing, not by individual test …
osterman Aug 15, 2026
8e02b00
fix(ci): give shard 1 real timeout headroom, it does far more work
osterman Aug 15, 2026
f146d93
fix(ci): make go test's own -timeout overridable, shard 1 still hit it
osterman Aug 16, 2026
2464f2b
fix(ci): balance shards by splitting non-tests packages across all 10
osterman Aug 16, 2026
3de6640
fix(ci): pin the heavy cmd package to shard 1, it stalled shard 4
osterman Aug 16, 2026
35f8d23
fix(ci): restrict shard 1's package invocation timeout, not both
osterman Aug 16, 2026
efdac9b
fix(ci): precompile cmd's test binary instead of budgeting more time
osterman Aug 16, 2026
549afd0
fix(ci): disable cgo for the cmd test binary precompile step
osterman Aug 16, 2026
38a8ebd
fix(ci): raise cmd.test execution timeout, windows needs more than 10m
osterman Aug 16, 2026
f7ef0d5
fix(ci): extract TestTerraformRegistryCache into its own parallel job
osterman Aug 17, 2026
c40e7bd
fix(ci): unblock and rebalance windows shard 1
osterman Aug 17, 2026
859e324
fix(ci): merge native Go coverage data
osterman Aug 17, 2026
75ca0b4
refactor(ci): move acceptance logic into scripts
osterman Aug 17, 2026
315fe16
fix(ci): rebalance Windows acceptance shards
osterman Aug 17, 2026
9fc6b04
Move acceptance CI orchestration to Mage
osterman Aug 18, 2026
fac3d2a
Fix Mage acceptance test environment
osterman Aug 18, 2026
fc1c382
Fix Windows acceptance shard failures
osterman Aug 18, 2026
c0208da
Address CodeRabbit review findings on PR #2940 acceptance sharding
osterman Aug 18, 2026
aa5c0f6
Raise internal/ci/acceptance patch coverage to 87%, fix patch-coverag…
osterman Aug 18, 2026
a69b23b
fix(ci): convert patch-test-coverage.sh indentation from spaces to tabs
osterman Aug 18, 2026
d53b829
Raise internal/ci/acceptance patch coverage to 96%, fix critical empt…
osterman Aug 18, 2026
b4af628
fix(security): resolve CodeQL allocation-size-overflow in mergeEnvSlices
osterman Aug 18, 2026
3314af3
Merge remote-tracking branch 'origin/main' into osterman/parallelize-…
osterman Aug 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 26 additions & 2 deletions .atmos.d/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -71,9 +71,9 @@ commands:
- type: shell
command: |
if [ "{{ .Flags.cover }}" = "true" ]; then
scripts/collect-coverage.sh "${TEST:-$(go list ./...)}" "${TESTARGS:-}"
go tool mage coverage:collect
else
go test ${TEST:-$(go list ./...)} ${TESTARGS:-} -timeout 40m
go test ${TEST:-$(go list ./...)} ${TESTARGS:-} -timeout "${GO_TEST_TIMEOUT:-40m}"
fi

- name: acc
Expand Down Expand Up @@ -143,6 +143,30 @@ commands:
command: go test -coverprofile=coverage.out ${TEST:-$(go list ./...)} ${TESTARGS:-}
- type: shell
command: go tool cover -html=coverage.out -o coverage.html
commands:
- name: download-shards
description: >-
Download every Linux shard's coverage artifact from this workflow
run in parallel (CI-only; collates before the Codecov upload
since Codecov aggregation happens from a single job).
steps:
- name: shards
type: matrix
# Keep this list in sync with .github/workflows/test.yml's
# env.TEST_SHARD_COUNT and the `test` job's matrix.shard.
matrix:
shard: ["1", "2", "3", "4", "5", "6", "7", "8", "9", "10"]
steps:
- name: download
type: shell
retry:
max_attempts: 3
initial_delay: 15s
backoff_strategy: constant
command: >-
gh run download "$GITHUB_RUN_ID"
-n "acceptance-coverage-linux-shard-{{ .matrix.shard }}"
-D "$GITHUB_WORKSPACE/coverage/shards/shard-{{ .matrix.shard }}"

- name: race
description: Run quick tests with race detector and shuffled order
Expand Down
127 changes: 90 additions & 37 deletions .claude/skills/test-coverage/scripts/patch-test-coverage.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,61 +28,114 @@ IGNORE_PATTERN='(^|/)mock_[^/]*\.go$|(^|/)mock/[^/]*\.go$|_test_helpers\.go$|(^|
touched_files="$(git diff --name-only --diff-filter=ACMR "${BASE_REF}...HEAD" -- '*.go' | grep -Ev "$IGNORE_PATTERN" || true)"

if [[ -z "$touched_files" ]]; then
echo "STATUS: NO_GO_CHANGES"
echo "No touched .go files vs ${BASE_REF} (after excluding mocks/testhelpers). Nothing to test."
exit 0
echo "STATUS: NO_GO_CHANGES"
echo "No touched .go files vs ${BASE_REF} (after excluding mocks/testhelpers). Nothing to test."
exit 0
fi

packages="$(
echo "$touched_files" | while IFS= read -r file; do
dir="$(dirname "$file")"
if [[ "$dir" == "." ]]; then
printf './\n'
else
printf './%s\n' "$dir"
fi
done | sort -u
dirs="$(
echo "$touched_files" | while IFS= read -r file; do
dirname "$file"
done | sort -u
)"

# Some touched directories belong to a *separate* Go module nested under the
# repo root (e.g. tools/lintroller, tools/gomodcheck each carry their own
# go.mod so they can be built by `go install`/`golangci-lint custom` without
# pulling the whole atmos dependency graph in). `go test ./tools/lintroller`
# from the root module fails immediately with "main module ... does not
# contain package ..." -- that's a false positive in this check, not a real
# test failure, so route those packages through their own nested `go test`
# invocation instead of the root module's package list.
root_packages=""
nested_module_roots=""
while IFS= read -r dir; do
walk="$dir"
nested_root=""
while [[ "$walk" != "." && "$walk" != "/" ]]; do
if [[ "$walk" != "." && -f "$walk/go.mod" ]]; then
nested_root="$walk"
break
fi
walk="$(dirname "$walk")"
done
if [[ -n "$nested_root" ]]; then
nested_module_roots="${nested_module_roots}${nested_root}"$'\n'
elif [[ "$dir" == "." ]]; then
root_packages="${root_packages}./"$'\n'
else
root_packages="${root_packages}./${dir}"$'\n'
fi
done <<<"$dirs"
packages="$(printf '%s' "$root_packages" | sort -u)"
nested_module_roots="$(printf '%s' "$nested_module_roots" | sort -u | sed '/^$/d')"

tmp_profile="$(mktemp "${TMPDIR:-/tmp}/atmos-patch-coverage.XXXXXX")"
tmp_output="$(mktemp "${TMPDIR:-/tmp}/atmos-patch-test-output.XXXXXX")"
cleanup() {
rm -f "$tmp_profile" "$tmp_output"
rm -f "$tmp_profile" "$tmp_output"
}
trap cleanup EXIT

set +e
# Go's default per-binary timeout is 10m, tuned for a single package on a
# dedicated runner. This branch's scoped package set includes slow
# acceptance suites (internal/exec, tests) that finish well under 10m on
# CI's dedicated runners, but on a shared dev machine running several
# worktree sessions at once (multiple `go test`/dev-server processes
# competing for the same CPUs -- confirmed via `uptime` load averages well
# above core count) the same suites can take several times longer, panicking
# the whole run with no actual test failure. -timeout 55m gives headroom up
# to just under CI's own Acceptance Tests job-level ceiling
# (.github/workflows/test.yml's `timeout-minutes: 60`) without masking a
# genuine deadlock outright.
# shellcheck disable=SC2086
go test -v -timeout 55m -coverprofile="$tmp_profile" -covermode=set $packages >"$tmp_output" 2>&1
test_exit=$?
test_exit=0
if [[ -n "$packages" ]]; then
# Go's default per-binary timeout is 10m, tuned for a single package on a
# dedicated runner. This branch's scoped package set includes slow
# acceptance suites (internal/exec, tests) that finish well under 10m on
# CI's dedicated runners, but on a shared dev machine running several
# worktree sessions at once (multiple `go test`/dev-server processes
# competing for the same CPUs -- confirmed via `uptime` load averages well
# above core count) the same suites can take several times longer, panicking
# the whole run with no actual test failure. -timeout 55m gives headroom up
# to just under CI's own Acceptance Tests job-level ceiling
# (.github/workflows/test.yml's `timeout-minutes: 60`) without masking a
# genuine deadlock outright.
#
# -tags mage lets magefiles/*.go (gated behind `//go:build mage`, see
# CLAUDE.md's Compilation section) build under this check; every other
# package in the repo is unaffected since they declare no such
# constraint. Without it, a touched magefiles/*.go reports a
# "build constraints exclude all Go files" [setup failed] false positive.
# shellcheck disable=SC2086
go test -tags mage -v -timeout 55m -coverprofile="$tmp_profile" -covermode=set $packages >"$tmp_output" 2>&1
test_exit=$?
fi

nested_output=""
if [[ -n "$nested_module_roots" ]]; then
while IFS= read -r nested_root; do
nested_output+=$'\n'"# nested module: ${nested_root}"$'\n'
module_result="$(cd "$nested_root" && go test -v -timeout 10m ./... 2>&1)"
module_exit=$?
nested_output+="$module_result"$'\n'
if [[ $module_exit -ne 0 ]]; then
test_exit=1
fi
done <<<"$nested_module_roots"
fi
set -e

echo "== Touched packages (vs ${BASE_REF}) =="
echo "$packages"
if [[ -n "$nested_module_roots" ]]; then
echo "Nested modules (own go.mod, tested separately, not part of the coverage profile below):"
echo "$nested_module_roots"
fi
echo

if [[ $test_exit -ne 0 ]]; then
echo "STATUS: TESTS_FAILING"
echo
echo "== Raw test output =="
cat "$tmp_output"
echo
echo "== Touched .go files (for in-scope vs pre-existing classification) =="
echo "$touched_files"
echo
echo "DISCLAIMER: scoped to touched packages' own tests only, not -coverpkg=./... breadth. A fast approximation for this patch, not a source of truth — CI's full-suite Codecov upload remains authoritative."
exit 1
echo "STATUS: TESTS_FAILING"
echo
echo "== Raw test output =="
cat "$tmp_output"
echo "$nested_output"
echo
echo "== Touched .go files (for in-scope vs pre-existing classification) =="
echo "$touched_files"
echo
echo "DISCLAIMER: scoped to touched packages' own tests only, not -coverpkg=./... breadth. A fast approximation for this patch, not a source of truth — CI's full-suite Codecov upload remains authoritative."
exit 1
fi

echo "STATUS: OK"
Expand Down
41 changes: 0 additions & 41 deletions .github/actions/setup-packer-retry/action.yml

This file was deleted.

Loading
Loading