Repository navigation
fix(sandcastle): Close the gaps Claude Review found after #236 merged - #242
Conversation
- Turn hooks and fsmonitor off for every git on the host through GIT_CONFIG_*, so Sandcastle's own `git worktree add` can't run a branch's agent-written post-checkout hook. A test adds a worktree with and without it. - Stop the whole run when an agent changed .git/config: the pipeline leaves its sandbox open instead of letting Sandcastle remove the worktree under that config, and main.mts rethrows the error. - Refuse to push commits whose messages or diffs hold a sandbox secret or a token shape, without quoting it on the issue. - Widen the PR warning to IsTestProject, Docker-backed packages, ProjectReference and deleted tests/ files, and compare with main's commit as origin reports it, not the agent-writable origin/main ref. - Rewrite closing references in the issue title (PR title and body) and in full issue URLs. - Show the planner the issues that stopped earlier in the run as blockers, alongside the ones in review. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Claude Review
This PR tightens Sandcastle's publish path. Main is now pinned to the commit git ls-remote reports, not the agent-writable origin/main ref. Hooks and fsmonitor are turned off through GIT_CONFIG_* for every git the host runs, Sandcastle's own included. The branch's commits are scanned for sandbox secrets before the push. If an agent changes .git/config, the run stops and the sandbox is left open. checkFileChanges also flags more MSBuild lines and deleted test files. Closing references are now rewritten in PR titles and in URL form. Issues that stopped earlier in the run are held back as blockers for the planner.
I traced the git-config path. It holds even though the inner publish catch swallows a GitConfigChangedError: stop() → commentOnIssue → gh re-asserts the config and throws again, so the error reaches the outer catch and nothing is posted. Pinning base by commit id, threading base through commitsAhead/checkFileChanges, and the planner and prompt changes all look right. Squash-only merging means agents' commit messages can't close issues, so rewriting only the title and body is enough.
Verdict: mergeable once the secret scan covers merge commits. That is a real hole in a gate meant to stop a public leak, and the fix is one flag. The other two are smaller gaps worth fixing or knowingly accepting. I couldn't run git here, so the git behaviour behind these findings is from git's documented defaults:
git log -pshows no diff for merge commits.git diffdetects renames by default.- Pathspecs are applied before rename detection.
I also couldn't check the Sandcastle source (node_modules isn't installed). The new comment in lib/shell.mts says Sandcastle passes the sandbox only what .sandcastle/.env names, so GIT_CONFIG_* never reaches it; I've taken that as given.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #242 +/- ##
=======================================
Coverage 85.63% 85.63%
=======================================
Files 75 75
Lines 1566 1566
Branches 186 186
=======================================
Hits 1341 1341
Misses 190 190
Partials 35 35 🚀 New features to boost your workflow:
|
…ests - The secret scan reads merge commits (--cc), so content written while resolving or amending a merge is checked, and scans only commit messages and added lines (-U0), so main's fake tokens in context or removed lines don't stop unrelated branches. - A file renamed away under tests/ is flagged like a deletion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Claude Review
This PR tightens the Sandcastle host in six ways. The host now merges and compares against main's commit ID from ls-remote rather than origin/main, which agents can move. Every pushed commit is scanned for sandbox secrets first. The check-file warning now also covers deleted tests/ files and more MSBuild lines. Hooks and fsmonitor are switched off for every git on the host through GIT_CONFIG_*. Closing keywords in issue titles and full GitHub URLs are rewritten. Issues that stopped earlier in the run are held back as blockers, and the run aborts when .git/config changes.
Verdict: the direction is right, but fix the first finding before merging. The handling for a changed git config misses its main case. The publish step's catch-all handler turns GitConfigChangedError into publish-failed. And a pipeline only skips close() if it saw the error itself. So Sandcastle's own git worktree remove can still run on the host under a config an agent wrote, which is exactly what this change is meant to prevent.
The secret scan has smaller gaps:
- It drops added lines whose content starts with
++. - It finds nothing if the host's git config forces colour.
- Replace refs (
refs/replace/*) in the agent-writable.gitcan hide content from the host'sgit log/git diff.
fetchMain's ls-remote pattern matches by suffix, so another branch can be taken for main. The check-file warning can still be avoided with multi-line edits to MSBuild files.
What I relied on without being able to check here: Sandcastle isn't installed in this checkout, so I couldn't read its close() or confirm that it doesn't pass the host environment into the container. Three points come from git's documented behaviour: the -G regex is extended, ls-remote patterns match on a ref's suffix, and pack-objects ignores replace refs.
Automated release blog posts for #242, opened by the release workflow. It holds every Release whose post isn't on main yet, rebuilt from main on each run. The [skip-release] title marker keeps its merge from starting another release. Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ecret scan (#244) ## Why PR Auto-Merge merged #242 before Claude Review finished with its final head. That review opened six threads on the hardening #242 added, and all six are real: - **A config change found while publishing was swallowed.** It turned into a `publish-failed` comment, and `sandbox.close()` still ran Sandcastle's git under the changed config. That also happened when another pipeline's agent changed the config after this pipeline's last host git call. - **The secret scan could miss added lines.** It skipped every line starting with `+++ `, which also drops an added line whose content starts with `++ `. It didn't scan file paths. With a user's `color.diff=always`, it saw no added lines at all. - **Replace refs could hide content.** `refs/replace` sits in the same writable `.git`, and `git log`/`diff`/`rev-list` honour it, but a push sends the real objects. - **`ls-remote origin refs/heads/main` could pick another branch.** It also matches a branch like `feature/refs/heads/main`, which sorts first. - **The check-file warning could still be dodged.** `discover_tests.py` and `needs-docker.sh` read whole XML documents. A `<!--`/`-->` pair or a `Condition` on its own line can drop a test project without changing any line the pattern matches. ## What changed - **`lib/build.mts`, `lib/shell.mts`:** the publish step rethrows `GitConfigChangedError`. `finally` checks the config again before it closes the sandbox, and throws `GitConfigChangedError` if it changed, whatever the pipeline returned (published, rejected, nothing to publish). The throw replaces the return value, so `main.mts` stops the run before Sandcastle's own git runs under the changed config. - **`lib/scan.mts`:** `addedLines` reads the patch with state. File headers run until a hunk header, and the number of `@`s gives the number of parent columns. A line is added when one of its columns is `+` and none is `-`, so an added `++ token` line is scanned. Header lines, and so the paths, are scanned too. `git log` gets `--no-color`. The commits are read with `--format=raw` rather than `%B`, so the author and committer names and emails, `encoding`, `gpgsig` and any extra header, which the push publishes too, are scanned. - **`lib/shell.mts`:** `HOST_GIT_CONFIG` adds `core.useReplaceRefs=false` for every git on the host, Sandcastle's included. The `git()` helper also passes `-c core.useReplaceRefs=false` itself, so the scan and the check-file warning hold up without `protectHostGit`. - **`lib/branches.mts`:** `mainCommit` keeps only the exact `refs/heads/main` line, and throws unless there is exactly one. - **`lib/check.mts`, `CODING_STANDARDS.md`:** every change is named for the MSBuild files under `tests/` and every `Directory.Build.*` file at any depth. The pathspecs carry explicit `:(glob)` magic (`:(glob)**/Directory.Build.*` and so on), so they match nested files under a host's `GIT_GLOB_PATHSPECS=1` too. The keyword pattern still covers MSBuild files elsewhere. ## Verification - `pnpm run test:sandcastle`: the type check is clean and 109 of 109 tests pass. New tests cover a config change found during publish and one found at the end of a pipeline that published, was rejected or had nothing to publish (each rejects with `GitConfigChangedError` and leaves the sandbox open), tokens in an author name and a committer email, a replace ref hiding a token and a check-file change (with and without `protectHostGit`, plus a control), nested `tests/Directory.Build.props` and `src/X/Directory.Build.targets` changes with and without `GIT_GLOB_PATHSPECS=1`, an added line that looks like a header, a scan with `color.ui=always`, an ls-remote result with a decoy branch, and a test project commented out on lines of their own plus a `Directory.Build.props` change. - The pre-push gate passed. Its final Claude Review round opened four threads; all four are fixed here, and `review:claude` is off. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: T <t@e.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Why
#236 replaced Sandcastle's local merger with one draft PR per issue. PR Auto-Merge merged it after its review cap, and Claude Review's pass over its final head then opened six threads. All six hold up:
git worktree addon the host too. That runspost-checkoutfrom the new worktree, which is the branch's copy of.github/hooks, and a branch can be one the agents wrote in an earlier run.GitConfigChangedErrorwas logged as one failed issue.sandbox.close()still ran Sandcastle's git under the tampered config, and the run could end with "All done."RequiresDockerchange was flagged. A Docker-backed package,IsTestProject=false, aProjectReference, or a deleted test project slipped through. The comparison also usedorigin/main, a ref the agents can move.What changed
lib/shell.mts,main.mts:protectHostGit()runs first. It addscore.hooksPath=/dev/nullandcore.fsmonitor=falsethroughGIT_CONFIG_COUNT/GIT_CONFIG_KEY_n/GIT_CONFIG_VALUE_n, after any entries already there. Every git process on the host inherits these, Sandcastle's included. They don't reach the sandbox: Sandcastle passes it only what.sandcastle/.envnames.lib/build.mts,main.mts: if a pipeline hitsGitConfigChangedError, it leaves its sandbox open and rethrows.main.mtsrethrows the first such error after the round, so the run exits non-zero with the warning.lib/scan.mts,lib/build.mts,lib/sandbox-env.mts: before the push, the host scans what it would publish, from main's commit to the checked commit. That is every commit message and every added line, fromgit log -p --cc -U0, so merge commits are read too. Context and removed lines are left out, so main's fake test tokens don't stop unrelated branches. The scan looks for the values.sandcastle/.envgives the sandbox and for Claude and GitHub token shapes. A match stops the issue assecret-in-commits, with a comment that doesn't quote it.lib/check.mts: the warning now covers MSBuild lines matchingRequiresDocker|IsTestProject|Testcontainers|Aspire.Hosting.Testing|Playwright|ProjectReference, and files deleted or renamed away undertests/.lib/branches.mts,lib/build.mts:fetchMain()returns main's commit asgit ls-remote originreports it. The merge,commitsAhead, the warning and the secret scan all compare with that commit id, not withrefs/remotes/origin/main.lib/publish.mts:withoutClosingKeywordsnow also rewrites the issue title, in the PR title and body, and fullgithub.com/.../issues/nand/pull/nURLs.lib/prompts.mts,plan-prompt.md: the planner's second list is now the held-back issues ({{HELD_BACK_JSON}}), both in review and stopped earlier in the run, and they count as blockers.Verification
pnpm run test:sandcastle: the type check is clean and 99 of 99 tests pass. The new tests run against real temp repositories: agit worktree addwithprotectHostGitdoesn't run apost-checkouthook, and the same command without it does.checkFileChangesflags a Testcontainers reference,IsTestProject=false, a deleted test project, a renamed test file andcheck.sh, but not an ordinarysrc/package change.publishedTextsees a secret added in a merge resolution, but not a fixture token in context. Other new tests cover the secret stop, the untouched sandbox on config tampering, and closing references in titles and URLs.🤖 Generated with Claude Code