Repository navigation
ci: save the large caches only from main - #758
Merged
Merged
Conversation
The repo's Actions cache sat at the 10 GB cap and evicted continuously. Entries are readable only from the saving ref and main, so each PR's own Go caches were dead weight that displaced main's warm ones. Go modules, Go build objects, the pnpm store and Playwright now save on main pushes and restore-only elsewhere; main restores gomod without restore-keys so superseded modules don't accumulate. housekeeping gains a cache-cleanup job on PR close that deletes the PR refs' caches. Refs #535 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…failed GOTOOLCHAIN=auto means max(local, go.mod), so a newer local Go (or runner image) builds and lints on a different toolchain than CI, and golangci-lint, built with go.mod's Go, panics on it. The Makefile now exports GOTOOLCHAIN derived from go.mod's go directive, and setup-env exports the same value for CI steps that run Go outside make. The setup-env comment that described auto as resolving go.mod exactly is corrected, and development.md records that bumping the go directive needs a golangci-lint built with at least that Go. pnpm-install ran with --reporter=silent, which prints nothing at all on failure, so a supply-chain policy violation surfaced as a bare Error 1. It now captures pnpm's default output and replays it only when the install fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Address pre-review: the PR-scoped churn came from PRs whose key missed main's, not every PR; two live generations are about 7.8 GiB so this refs #535. Cleanup gets its own concurrency group and tolerates a failed delete; pnpm prune runs on main only; docs and comments follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ten tests in tests/integration bring up servers or containers of their own (a ClickHouse, a NATS server, a Redis, processes on random ports) and spend most of their time waiting on leases, handovers and restarts. Run serially they made up most of the package's 337s on a 4-core machine. Marked t.Parallel, they overlap each other and the multi-process roles test, which already ran in parallel. Each was checked for shared state first: per-test tables and databases on the shared ClickHouse, no t.Setenv, no captured default logger, no fixed ports. TestShardClaims_ScaleOneThreeTwo and the tests that share the wired app stay sequential. Refs #701 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stop pnpm's node_modules purge prompt from hanging the captured install, document the GOTOOLCHAIN override and the golangci-lint minor-version coupling, and update the CI comments that still described GOTOOLCHAIN=auto. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pre-push review gate could attest to code no reviewer read. review-marker.sh keyed the marker to whatever HEAD the session's launch directory had when the reviewer stopped, so a commit made while a reviewer ran inherited its ship_it, and a review of a sibling worktree marked the launch directory's branch instead. Reviewers now end with `REVIEWED: <sha>` naming the commit they pinned before reading anything. The hook writes the marker into each worktree on that commit, and only when that worktree's HEAD reflog shows it on the commit from the start of the review (the subagent transcript's first timestamp) until now. Same-commit entries such as `git reset` or `git stash` don't count as moves. A report delivered through the subagent hand-back leaves only closing text in last_assistant_message, so the hook reads the hand-back's message from the transcript. Every decision goes to stderr and to tmp/review-marker.log, which the push gate prints when it blocks. agent-bash-gate.sh checked HEAD of the session's launch directory for every push. It now follows the push to where it runs (the tool call's cwd, a preceding cd/pushd, git -C), resolves each refspec source to the commit it publishes, and looks for that commit's markers in every worktree of the repository. It blocks when it can't tell: a computed directory or refspec, GIT_DIR, --all/--mirror, a glob refspec, a wrapped push, or a payload without cwd. A push into another repository with no reviewer manifest is outside the policy. Pushes through `git -C` and from a detached HEAD were not gated at all before. skip-pre-push-review.sh writes to the worktree root when run from a subdirectory. .claude/hooks/review-gate.test.sh drives both hooks with synthetic events in a scratch repository with sibling worktrees; it is a make verify leaf (test-review-gate). Because the pre-commit hook runs it with GIT_DIR exported, it clears git's repository variables, names every repository with -C, and refuses to write unless git sees the scratch directory as outside any repository and creates each scratch repository where asked. /prepush, the reviewer agents, AGENTS.md and the Claude Code docs page describe the REVIEWED line and the rule not to move HEAD while reviewers run. Closes #454 Closes #565 Closes #659 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Integration job had grown to about 8 minutes and was CI's critical path. make test-integration now runs two parts, test-integration-app (tests/integration, with -parallel 8 since its parallel tests mostly wait) and test-integration-backends (internal/cache, natsspike and internal/mq's tagged tests), at once. CI runs one matrix job per part: the changes job reads INTEGRATION_PARTS from the Makefile (scripts/ci/integration-parts.sh) and hands the coverage job the fragments and producer jobs to wait for, so a part added to the Makefile gets its own job, and CI and a local run execute the same commands. Each part keeps its own build-cache suffix, so no part saves a cache holding only its own test binaries. internal/mq's integration tests were chosen by a -run name pattern, so a new prefix would have been skipped silently. scripts/ci/tagged-tests.sh run selects the tests that only build with the integration tag, and tagged-tests.sh check, run before every part, fails while a package with tagged tests belongs to no part. tagged-tests.test.sh checks the selection against go test -list on a throwaway module (a verify leaf). natsspike stays in the suite: it pins the nats-server behavior the external NATS topology rests on, and it runs beside internal/cache. Closes #701 Closes #692 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move the close-time cleanup into its own workflow (no always-skipped check, no concurrency trick), make it fail when a delete fails, and correct what a PR still saves (CodeQL, golangci, Astro). Docs: measurements move to the sizing section, units and stale sizes refreshed, prune comment no longer overclaims, AGENTS.md and the dev guide describe the new policy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The cheap filter in agent-bash-gate.sh strips quotes one line at a time, so a heredoc body, a multi-line quoted string or a comment that mentions `git push` reached the tokenizer, which correctly found no push, and the gate then refused the command as one it couldn't follow. It now refuses only when the tokens hold a `git … push` the walk didn't gate: behind an unknown wrapper, or inside a command substitution. The gate runs before the command line does, so in `git commit … && git push` it judged the commit before the new one. A push that follows a HEAD-moving git command (commit, merge, rebase, reset, checkout, switch, cherry-pick, revert, am, pull) on the same line is now blocked with "push in a separate command". Pushes inside if/while/until/for … do are followed instead of refused, and the matching refspec `:` (or `+:`) is blocked like --all instead of being read as a delete. review-marker.sh uses the hand-back message whenever the final message lacks either the VERDICT or the REVIEWED line, retries the transcript read once, and resolves an abbreviated REVIEWED sha that names exactly one commit. A ship_it without a REVIEWED line, which is what a session started before this change produces because reviewer definitions load at session start, logs how to recover, and the gate's block message says the same and now shows the latest decisions from every worktree's log. pre-push-reviewer pins its commit in step 1 with `git -C <path> rev-parse HEAD`. The docs say the gate checks the commit each refspec publishes (its tip, or HEAD with no refspec), and the remaining copies that described a VERDICT-only, HEAD-keyed marker now describe REVIEWED plus VERDICT and the reviewed commit. The test covers each change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… coupling Prefer go.mod's toolchain line, require x.y.z, document that a new minor needs a matching golangci-lint (it refuses to load its config, not panics), the go bump command, the pnpm prompt, and the Go prerequisite row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review follow-ups to the integration split. - scripts/ci/integration-parts.sh read only the first INTEGRATION_PARTS line, so a later `+=` or a second assignment changed what make runs but not CI's matrix or the coverage job's wait list. It now refuses anything but a single `INTEGRATION_PARTS := ...` line, and its --check mode, run by check-integration-parts before every part, fails when its list differs from make's own. integration-parts.test.sh covers the shapes it must refuse (a verify leaf). - scripts/ci/tagged-tests.sh missed tagged examples and tests whose signature wraps across lines. It now takes every top-level func named Test*, Fuzz* or Example*, and its test compares the selection with every entry `go test -list` prints, with an example and a wrapped signature among the fixtures. - Docs: AGENTS.md's CI summary, the ci.yml comments that still called e2e the long pole or counted three fragments, development.md's account of the parts and of which tests run in parallel, where a package with both unit and tagged tests goes, a fresh cache suffix per job in the README's add-a-job recipe, estimates flagged in the timing table, and a transition note for purging the retired -integration cache entries. Refs #701 Refs #692 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # Makefile # docs/src/content/docs/development.md
# Conflicts: # .github/workflows/README.md
# Conflicts: # .github/actions/setup-env/action.yml # .github/workflows/README.md # AGENTS.md
agent-bash-gate.sh looked for an unfollowed push only when it had followed none, so a second push on the same line behind a wrapper or in a command substitution went unchecked once the first was gated. The walk now records each simple command it gates, and every other one is checked. GIT_DIR or GIT_WORK_TREE exported, declared or assigned earlier on the line now makes the push's repository unknown, the same as the inline form. The test requires jq up front, and development.md lists jq among the prerequisites, since make verify now runs it. pr-review-locally names the PR worktree's path in its reviewer prompts, because reviewers pin the worktree their prompt names, else their current directory. The reviewer agents say an unresolvable REVIEWED sha means no marker, and the remaining "marker for HEAD" phrasings say "for the commit being pushed". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A PR saves its own CodeQL dependency cache whenever main has no entry for the key, including after main's entry was evicted; the docs now say so with dated figures. Also restores the base wording of the housekeeping sentence in AGENTS.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Makefile and setup-env each derived the pin and had already drifted. scripts/ci/go-toolchain.sh now owns the derivation and the x.y.z check, with a fixture test as a verify leaf; the Makefile passes the pin to its parse-time go list, and development.md's Go version hints match. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The missing-suffix error, the input description and the header comment still allowed reusing another flavor's suffix when the flags match. Two jobs on one key race to save it, which is why the integration parts each take their own. Also restores the blank line the merge dropped before the test-review-gate rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The #535 sentences had landed between the two-generation budget and the sentence it explains, so "cached once" read as a consequence of PRs no longer saving. Move them to the end of the paragraph, and give the CodeQL row one date and an order for "on main" and "gone". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The parse-time go list used the derived pin directly, so `make GOTOOLCHAIN=local` still downloaded go.mod's toolchain for it. $(GOTOOLCHAIN) equals the pin unless overridden on the command line, and passing it explicitly still covers Make before 4.4, whose $(shell) doesn't see exported variables. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Four gaps in the push gate, each now covered by review-gate.test.sh: - Every check fed the command to `grep -q` through a pipe under pipefail. Past a pipe buffer (64 KB), grep's early exit sent SIGPIPE to the writer and the pipeline read as no match, so a long `gh pr create` without --draft, or a push ahead of a long heredoc body, went unchecked. The checks now read here-strings. - An exported GIT_DIR or GIT_WORK_TREE shared a variable with "directory unknown", so a later absolute cd, pushd or git -C cleared it. It is tracked on its own now; only unset or the end of a subshell clears it. - A HEAD-moving git command counted only when run directly, so `timeout 60 git commit … && git push` or `out=$(git commit …) && git push` had the push judged on the commit before the move. A wrapped or substituted one counts too. - A push in code handed to a shell (bash -c, sh -c, eval, a heredoc or here-string read by a shell) was never seen, and one in a process substitution was dropped as a redirect target. The first now blocks as a push the gate can't follow; the second is followed like any other push. The Claude Code docs now say what a gating reviewer must end its report with for its ship_it to write a marker, matching AGENTS.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three more gaps in the push gate, each covered by review-gate.test.sh: - A substitution in a here-string or a redirect target was dropped with its word, so `grep -q x <<< "$(git -C ../wt-b push)"` or `echo hi > "$(git push …)"` went unchecked, and a commit in one didn't count as a HEAD move. The tokenizer now keeps the text of every $(…) and `…` wherever its word goes, and the push and HEAD-move checks scan all of it. A quoted substitution no longer slips past the cheap pre-filter either. - Inside $(…), a here-string's text was glued to `<<<`, so a push in `$(sh <<< 'git push')` didn't read as one. - Code handed to a shell was scanned without joining backslash-newline continuations, so a push split across lines in `bash -c` or a heredoc read by bash went through. AGENTS.md, the Claude Code docs and the changelog now say the gate reads only the command line, list what it blocks, and name the forms it doesn't follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The last change scanned the text of every $(…) for a push, but with its quotes already stripped, so `n=$(grep -c 'git push' AGENTS.md)`, `git commit -m "$(echo 'explain the git push gate')"` and the like were refused as pushes the gate can't follow. The tokenizer now marks quoted text inside a $(…), and each substitution is checked with that text replaced by a placeholder; only a substitution that hands it to a shell (`$(bash -c '…')`, `$(sh <<<'…')`, `$(eval '…')`) is checked in full. Two forms the wider pre-filter now tokenizes failed to parse: a heredoc inside $(…) closed with `EOF)"` on one line, and an `$'…'` string holding `\'`. Both parse now; an apostrophe in a comment inside $(…) still doesn't, and the docs say such a line is refused. The test for the here-string spacing inside $(…) now uses `<<<'…'` with no space, so it fails without that fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two code paths from the last change had no test: stripping the quote
markers from a word's value, and reading an $'…' string inside $(…).
Each now has cases that fail when it is removed: a substitution's output
handed to eval or bash -c, and an $'…' string in a $(…), pushed or not.
The marker strip also runs in one pass with tr. Under bash 3.2,
${_w//…} took 47 s on a 200 KB substitution holding 200 quoted strings,
past the hook's time limit; it now takes about a second.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts: - docs/src/content/docs/development.md: kept origin/main's reworded make vulncheck row and this branch's make verify row (adds review-gate hook tests). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Conflicts: - docs/src/content/docs/development.md: make vulncheck row from the merged-in main (govulncheck -scan package); make verify row from this branch, which already carries the review-gate hook tests plus the integration-split fixtures. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Conflicts: - docs/src/content/docs/development.md: make vulncheck row from the merged-in main (govulncheck -scan package); make verify row from this branch, which carries the go-toolchain fixtures on top of the integration-split fixtures. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
github-actions
Bot
added
the
documentation
Member
Author
|
@coderabbitai review |
|
Member
Author
|
@coderabbitai review |
|
Member
Author
|
@coderabbitai review |
|
Member
Author
|
@coderabbitai review |
|
Member
Author
|
@coderabbitai review |
|
Member
Author
|
@coderabbitai review |
✅ Action performedReview finished.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
EricAndrechek
marked this pull request as ready for review
October 7, 2026 22:56
EricAndrechek
dismissed
coderabbitai[bot]’s stale review
October 8, 2026 19:24
The base branch was changed.
|
📚 Docs preview is live → https://72ff9891-wavehouse-docs.wave-rf.workers.dev
|
Contributor
EricAndrechek
added a commit
that referenced
this pull request
Oct 8, 2026
4 tasks done
This was referenced Oct 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
main's. A PR saved its own ~2.4 GB set only when its key missedmain's (it changed go.mod/go.sum/pnpm-lock.yaml, or ran between amainbump andmain's save); measured 2026-10-07, all 6.1 GiB of PR-scoped entries (18) came from one such PR, against 3.86 GiB (16) onmain.setup-env: the Go module and build caches, the pnpm store and the Playwright cache now save only onmain(actions/cache) and are restore-only elsewhere (actions/cache/restore: PRs, merge-queue refs, branches).mainrestoresgomod-v1without restore-keys so superseded modules stop accumulating, and the docs job'spnpm store pruneruns only onmain, where its result can be saved. golangci and Astro caches are unchanged (about 40 MB and a few KB).pr-cache-cleanup.yml: on PR close it deletes the caches under the PR's refs (actions: writeonly, no checkout, nothing from the PR executed; fails if either delete fails). It covers what a PR still saves: GitHub's CodeQL dependency cache (about 0.4-0.6 GiB, whenevermainhas no entry for its key) and the small golangci/Astro entries..github/workflows/README.md, AGENTS.md,development.mdand the CHANGELOG describe the cache rules, inventory and sizing policy to match.Test plan
make cigreen locally;make lint-gha(actionlint) clean.ship_it; docs reviewship_it.Post Run actions/cachesave lines for the Go, pnpm or Playwright caches; the restore steps (actions/cache/restore) show a hit or restore-key partial hit forgomod-v1,gobuild-v3-*,pnpm;gh cache listshows no newrefs/pull/<n>/mergeGo entries.mainrun after merge:gomod-v1, eachgobuild-v3flavor andpnpmrestore then save on a key miss, exact hits save nothing; repo usage trends toward about 4 GB.gh cache delete --all --ref refs/pull/<n>/merge(and/head) once for any large leftover set.PR cache cleanupworkflow'sDelete PR cachesjob passes andgh cache list --ref refs/pull/<n>/mergeis empty. It can only be tested after merge (close any PR, then check both refs).Remaining findings: none.
Related Issues
Refs #535 (not closed: two live generations on
mainare about 7.8 GiB of 10, inferred, above the ~70% target; this removes the PR-scoped churn, not all the pressure). Part of #740.Stacked on #757 (base
build/pin-gotoolchain)🤖 Generated with Claude Code