Skip to content

Commit dc759f2

Browse files
ci(validate-deps): judge a pull request's OSV findings against the merge base; main keeps the absolute daily scan; job timeout 15 min (#22138)
Fixes #22082 Clause-②: no ## What this does Implements ruling A as recorded on the card (comment 6049727506). On `pull_request`, the OSV verdict of `validate-deps.yml` is now base-relative. On `schedule` and `workflow_dispatch`, `main` keeps the absolute verdict from the same step as before. That step's only change is `if: github.event_name != 'pull_request'` (its version comment is also corrected, see below). The job gains `timeout-minutes: 15`, sized from the measured p50 of 1.3 min and max of 1.9 min. - **Lockfile unchanged.** The new stage step compares the pull request's test merge with its merge base. If neither `pnpm-lock.yaml` nor `osv-scanner.toml` differs, the step prints the notice "lockfile unchanged — inherits main's verdict" and runs no scan. The ledger counts as part of the input because a PR that deletes an exemption changes the verdict without touching the lockfile (replay R3 below). - **Otherwise, both lockfiles are scanned.** The same pinned action scans the PR's lockfile and the merge base's. Each is judged under its own `osv-scanner.toml`, so the merge base's verdict is what `main`'s scan gives and the PR's verdict is what the old absolute scan gave. `scripts/osv-base-relative.mjs` then sorts the advisories: - An advisory matched by both is **inherited**. It gets a `notice` annotation that names it, the package versions each side matched, and the open finding card that names it, when one exists. It does not fail the step. - An advisory matched only by the PR's lockfile is **introduced**. It gets an `error` annotation and fails the step, as before. - **The merge base.** `actions/checkout` puts the PR's test merge (`refs/pull/N/merge`) at depth 1. Its first parent is the base-branch commit that GitHub merged the PR into, which makes it the merge base of the scanned tree. The step reads that parent from the commit object and fetches it with one depth-1 `git fetch` (never `fetch-depth: 0`). If HEAD is not a two-parent merge, the step fails with an error annotation. - **Fail-closed.** A side is read only when its result file parses and its step outcome matches its finding count (success means zero findings, failure means some). In every other case the step fails and names that side. This adds no new gate, changes nothing that `main`'s scan judges, and adds no exemption. `osv-scanner.toml`, `scripts/check-osv-exemptions.mjs` and the other gate scripts are untouched, and they still run first and unchanged. ## Why a helper script, and why not upstream's `osv-reporter-action` The upstream reporter was read at `google/osv-scanner` v2.5.0 (`cmd/osv-reporter/main.go`, `internal/ci/vulnerability_result_diff.go`). Three things rule it out: - **It fails open.** When the new-side result file is missing or does not parse, it logs a warning, treats the side as having no findings and exits 0. Upstream's own reusable PR workflow runs both scans with `continue-on-error: true`, so a scan error on the PR side passes the gate. - **It needs both sides at the same source path.** The diff keys on `source.path`, so the base lockfile would have to be swapped into the workspace and restored mid-job. - **It cannot name the inherited advisories**, and the ruling requires that notice. The helper is one definition of introduced versus inherited, with the scan-error guard built in. Its `--self-test` (23 cases, floor pinned) runs as the first line of the stage step, so `check:self-test-wired` sees it. The helper is in the `paths:` filter, so a PR that edits it runs this workflow. ## Measured mechanism (osv-scanner 2.5.0, the binary in the pinned action's image) - **The pin.** `f4cfcc01edc9c8b756a9b873b7a623ca674da51e` is an untagged upstream commit between the v2.5.0 and v2.5.1 tags. Its `action.yml` runs `ghcr.io/google/osv-scanner-action:v2.5.0`, and run 37483796939's job steps include "Pull ghcr.io/google/osv-scanner-action:v2.5.0". The `# v2.3.8` comment had been stale since Dependabot's sha bump (#9209). It is corrected, and all three `uses:` lines stay on that one sha. - **The image's binary matches the release binary.** It is built from the same `cmd/osv-scanner` with the same flags (goreleaser builds `osv-scanner-action`). The release binary `osv-scanner_linux_amd64` matched its published SHA256SUMS (`edcfc41d…`). The action's entrypoint is upstream's `exit_code_redirect.sh`, which rewrites exit 128 (no packages) to 0. - **The ledger is found only in the lockfile's own directory.** With a ledger one directory up, an ignored advisory was still reported. The workflow therefore passes `--config` explicitly on both sides. This settles dispatch assumption ②: without `--config`, a base lockfile staged at another path is scanned with no ledger at all. - **Exit codes, with `--config` and `--output-file`:** | case | exit | result file | |:--|:--|:--| | no findings | 0 | written, `results: []` | | findings | 1 | written | | expired `ignoreUntil` | 1 | written; the advisory is reported again (the ledger's expiry convention holds on both sides) | | unknown key / broken TOML / missing `--config` file | 127 | **none** | | missing lockfile | 127 | **none** | | no packages | 128, rewritten to 0 by the action | **none** | Because the outcome alone cannot tell findings from errors, the helper's file-plus-outcome agreement check is what keeps the verdict closed. ## The pin The body is written once, when the PR is created and before any run exists, so the run ids are posted in the `os-dev-report` comment on #22082, keyed by the head shas named here. **Real runs** (seat amendment, from the dev's report on #22082). All are `validate-deps.yml` on `pull_request`, against the merge base `ef1fcb26a24f` (the `main` tip): | Head | Run (job) | Result | |---|---|---| | `ce598b1ab7` | `37719042481` (`113122177520`) | success, with the "lockfile unchanged" notice | | `501569be49` (probe: `minimist` 1.2.5) | `37719190017` (`113122655549`) | **failure**, one error: `GHSA-xvch-5gv4-984h` (minimist@1.2.5) | | `3576cdc334` (probe reverted) | `37719368398` (`113123226526`) | success, with the "lockfile unchanged" notice | | `5473cad514` (probe: `minimist` 1.2.8, no advisory, lockfile changed) | `37719558844` (`113123838149`) | success, with **six inherited notices** for `next@16.3.6` | | `cc98faf47a` (final; same tree as `ce598b1ab7`) | `37719733046` (`113124391631`) | success, with the "lockfile unchanged" notice | `main` at `ef1fcb26a24f` carries those six advisories (filed as #22148). So: - the inherited pin is a **real run**: `37719558844`; - the `scripts`-only pin is `37719368398` / `37719733046`, against that advisory-carrying `main`; - the replays R1–R3 below are supplementary. ### "A lockfile that introduces a new advisory fails": real run - **Head `501569be49`** is a TEMPORARY probe commit. It adds root devDependency `minimist` 1.2.5 (GHSA-xvch-5gv4-984h, fixed in 1.2.6) through `pnpm add -D -w`. The PR squashes, and `3576cdc334` reverts the probe. - **Expected run result:** the stage step reports `compare=true`, both scans run, and the judge fails with exactly one `error` annotation, for GHSA-xvch-5gv4-984h (minimist@1.2.5). - **Pre-check run locally**, with the same steps against main `8fc50b7647` and the 2026-10-06 database snapshot described under R1–R3: ``` ::error title=OSV advisory introduced by this pull request::GHSA-xvch-5gv4-984h (minimist@1.2.5) is matched by this pull request's lockfile and not by the merge base's (8fc50b7). ... OSV base-relative verdict against the merge base (8fc50b7): 1 introduced, 0 inherited, 0 resolved. judge exit=1 ``` ### "Lockfile unchanged": real runs - The heads are `ce598b1ab7` (the implementation, at PR open) and `3576cdc334` (after the probe's revert). The two trees are identical: both are `d477febb20`. - This PR changes neither `pnpm-lock.yaml` nor `osv-scanner.toml`, and `.github/workflows/validate-deps.yml` is in the `paths:` filter, so the workflow runs on it. - **Expected:** a pass with the notice "lockfile unchanged — inherits main's verdict", and the three scan and judge steps skipped. - This branch runs no scan, so `main`'s advisory state cannot reach it. That is why a `scripts`-only PR against a `main` that carries an advisory passes here. R1 replays exactly that case. ### "Inherited passes with a notice": real run `37719558844` (above); the replays below are supplementary `main` is clean today: scheduled run 37565270565 succeeded. A real PR run here cannot have a merge base that carries an advisory unless `main` is touched or a throwaway branch is made, and both are out of bounds. Following the dispatch, the step logic was replayed on real historical pairs instead. The replay worked like this: - It built the test merge GitHub would build (`git merge-tree --write-tree BASE PRHEAD`, then `git commit-tree -p BASE -p PRHEAD`) in an isolated clone. - It ran the stage step's `run:` text verbatim, as extracted from this branch's workflow. The only change was the helper's path, because the historical trees carry no copy of the helper. - It ran each scan step's `scan-args` verbatim through upstream's own `exit_code_redirect.sh` and the release binary, with outcome = exit 0 ? success : failure. - It ran the judge step's `run:` text verbatim. **The gap, named:** this container's egress policy refuses `api.osv.dev`. The scans therefore ran `--offline` against a local snapshot of the OSV npm database. That snapshot is an `all.zip` of 230,017 records, with max `modified` 2026-10-06T16:30:04Z and sha256 `0bd50081c140…`. It was found already on this container's disk, and its provenance is otherwise unknown. Its GHSA-wq5f-xc86-pv6w (sharp, published 13:43Z) and GHSA-pqg4-j6r4-53mv (shell-quote, published 13:40Z) records are those finding card #22013 measured. The anchor lookup used the real repo-scoped listing. **R1: PR #22002 as run 37483796939 saw it (the card's specimen: a `scripts`-only change).** Base is `f0022c46c1`, `main`'s tip when the run was created (15:00:57Z). PR head is `95c510eb8a`. ``` osv-base-relative self-test: 23 case(s), all held (floor 23) ::notice title=OSV-Scanner verdict (base-relative)::lockfile unchanged — inherits main's verdict. pnpm-lock.yaml and osv-scanner.toml are identical to the merge base f0022c4, so this pull request introduces no advisory; main's daily scheduled scan judges that lockfile. GITHUB_OUTPUT: base=f0022c46c10a142f5fb29e6891a4fc06ec05a296 / compare=false -- for contrast, the old absolute step on the same test merge's lockfile: | https://osv.dev/GHSA-wq5f-xc86-pv6w | 8.9 | npm | sharp | 0.35.4 | 0.35.5 | pnpm-lock.yaml | | https://osv.dev/GHSA-pqg4-j6r4-53mv | 9.2 | npm | shell-quote | 1.10.0 | 1.11.0 | pnpm-lock.yaml | Exit code: 1 ``` **R2: PR #21951 (a real lockfile-changing PR against a `main` that carries advisories).** Base is `4c49150e0c`, PR head is `539b0752cd`. The constructed merge's `pnpm-lock.yaml`, `osv-scanner.toml` and `pnpm-workspace.yaml` blobs equal those of the PR's real squash commit `787104baa9`. ``` GITHUB_OUTPUT: base=4c49150e0cb27a493390c4ac24e0b93e13efe662 / compare=true osv-scan-base: found 1372 packages; GHSA-hp3w-g68c-fv3c filtered (the base's ledger); Exit code: 1 -> outcome failure osv-scan-head: found 1371 packages; GHSA-hp3w-g68c-fv3c filtered (the PR's ledger); Exit code: 1 -> outcome failure ::notice title=OSV advisory inherited from the merge base::GHSA-pqg4-j6r4-53mv (here: shell-quote@1.10.0; at the merge base: shell-quote@1.10.0) is matched by the merge base's lockfile (4c49150) too, so it is main's to fix and does not fail this pull request; no open issue names it yet; main's daily scheduled scan is the signal that files its finding card. ::notice title=OSV advisory inherited from the merge base::GHSA-wq5f-xc86-pv6w (here: sharp@0.35.4; at the merge base: sharp@0.35.4) is matched by the merge base's lockfile (4c49150) too, so it is main's to fix and does not fail this pull request; no open issue names it yet; main's daily scheduled scan is the signal that files its finding card. OSV base-relative verdict against the merge base (4c49150): 0 introduced, 2 inherited, 3 resolved. resolved GHSA-238p-pmpm-9mq7 katex@0.16.47 resolved GHSA-68fv-2mgg-jv7q source-map-js@1.2.1 resolved GHSA-jqcg-44mw-7w3h proxy-addr@2.0.7 judge exit=0 -- the old absolute step on the same lockfile: sharp + shell-quote, Exit code: 1 ``` The anchor clause reads "no open issue names it yet" because #22013 is closed today. A live probe of the same listing found the open issues that name a given advisory id. It also turned up a seat board post quoting one, which is why board posts (`pm:seat`) are skipped; the self-test pins both the anchor case and the board-post skip. **R3: a synthetic ledger-only PR** that deletes the sprintf-js exemption from `787104baa9`'s `osv-scanner.toml` and touches no lockfile: ``` GITHUB_OUTPUT: base=787104baa9ecca77ff13a8e83a4b7349728feadc / compare=true (no short-circuit: the ledger differs) ::error title=OSV advisory introduced by this pull request::GHSA-hp3w-g68c-fv3c (sprintf-js@1.1.3) is matched by this pull request's lockfile and not by the merge base's (787104b). ... OSV base-relative verdict against the merge base (787104b): 1 introduced, 2 inherited, 0 resolved. judge exit=1 ``` ## Helper ablations (each run with `scripts/ablation-replace.mjs`, which restores the file and confirms the restored blob equals HEAD) - **A1: `compare` ignores the base** (`if (base.has(id)) inherited.push` becomes `if (false) inherited.push`). The self-test fails: 25 failed expectations over 23 cases, exit 1. - **A2: a missing result file read as "no findings"** (upstream's fail-open). The self-test fails: 4 failed expectations, exit 1. - **Restore:** the blob is back to `f35d31ee1072` = HEAD, and `git diff HEAD` is empty. ## Local verification These gates ran at `bf15ed3831`, plus `ce598b1ab7` for the entry-guard fix. The final tree `d477febb20` is identical at `ce598b1ab7` and `3576cdc334`. - **Derived gates:** `node scripts/pm/dispatch-gates.mjs --commands` derived 54 commands over the actual diff, and all 54 exit 0. - **One red caught and fixed:** `check:entry-guard` first went red because the helper exported bindings and ran on import. The fix puts the dispatch behind `isEntrypoint`, and the gate is now green. - **`pnpm check:pm-dispatch-gates`:** 1976 cases pass, in 820.5s. - **`--ran` reconciliation:** 54 derived, 54 run, 0 NOT-MEASURED. - **eslint on the new file:** `npx eslint --no-inline-config --format json scripts/osv-base-relative.mjs` reported 1 file, 0 errors and 0 warnings. - The file is in eslint's population: a JSON result came back for it, not an "ignored" warning. - The config enables no type-aware linting (`eslint.config.mjs` has no `parserOptions.project`), so this diff cannot change the verdict on any file it does not touch. - The whole-repo `pnpm lint` belongs to CI. ## Acceptance notes - **The unit is the advisory id,** following the ruling's wording ("an advisory matched by both is reported as inherited"). Suppose a PR adds a second vulnerable version of a package for an advisory `main` already carries. That advisory reads as inherited, and the notice lists both sides' package versions so the addition is visible. `main`'s daily scan still judges it. Upstream's reporter counts occurrences instead. - **Prose that drifts with this change, in files outside this PR's allowed surface.** Both are out of bounds for this PR, and this PR has no carrier for them: - The `osv-scanner.toml` header says the workflow "blocks on any advisory at any severity". That is still true of `main`'s scan, but a PR now answers only for what it introduces. - The `osv-scanner.toml` and `scripts/check-osv-exemptions.mjs` headers cite measurements "against v2.3.8". The pinned image has been v2.5.0 since the Dependabot bump. - **Remaining inherited-red path, by design:** an exemption that expires on `main` still reddens every dependency-touching PR through the unchanged "Verify OSV exemptions carry an expiry and a reason" step, which the ruling keeps as is. - **The job keeps `issues: write`,** which no step in it uses. The judge's anchor lookup only reads. Permissions are unchanged here. --- _Generated by [Claude Code](https://claude.ai/code/session_01VF48aw8RPG6wzDnMgp6rtw)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent d0bb78e commit dc759f2

2 files changed

Lines changed: 690 additions & 1 deletion

File tree

‎.github/workflows/validate-deps.yml‎

Lines changed: 108 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,9 @@ on:
1717
# that governs it ever running on the PR that adds it.
1818
- 'osv-scanner.toml'
1919
- 'scripts/check-osv-exemptions.mjs'
20+
# The pull_request verdict of the OSV step (#22082): an edit to it must
21+
# be exercised on the PR that makes it.
22+
- 'scripts/osv-base-relative.mjs'
2023
# Re-run when the workflow itself changes, so edits to these gates are
2124
# exercised on the PR that introduces them.
2225
- '.github/workflows/validate-deps.yml'
@@ -32,6 +35,11 @@ jobs:
3235
validate:
3336
name: Validate Package Dependencies
3437
runs-on: ubuntu-latest
38+
# Measured over the 100 most recent completed runs: p50 1.3 min, max
39+
# 1.9 min (#22082). 15 leaves room for a cold pnpm store and two OSV
40+
# scans on a pull request, and stops a hung step long before GitHub's
41+
# 360-minute default would.
42+
timeout-minutes: 15
3543
permissions:
3644
contents: read
3745
issues: write
@@ -138,12 +146,111 @@ jobs:
138146
node scripts/check-osv-exemptions.mjs --self-test
139147
node scripts/check-osv-exemptions.mjs
140148
149+
# Two verdicts, one scanner pin (#22082, ruling A).
150+
#
151+
# `schedule` and `workflow_dispatch` judge `main` ABSOLUTELY: every
152+
# advisory the lockfile matches fails the step. That daily red is the
153+
# signal that files the finding card, and this step is unchanged by the
154+
# pull_request verdict below.
155+
#
156+
# A pull request is judged BASE-RELATIVELY: it answers for the advisories
157+
# it introduces, not for the ones `main` already carries -- those are
158+
# `main`'s to fix, and a pull request that changes no dependency cannot
159+
# fix them. The steps below scan the pull request's lockfile and the merge
160+
# base's, each under its own `osv-scanner.toml`, and
161+
# scripts/osv-base-relative.mjs fails only on an advisory the merge base
162+
# does not match; the others become notices naming the advisory and its
163+
# open finding card. A scan that does not reach a verdict fails closed.
164+
#
165+
# The pin is an untagged upstream commit after the v2.5.0 tag whose
166+
# action.yml runs the ghcr.io/google/osv-scanner-action:v2.5.0 image (the
167+
# comment here used to read v2.3.8). Keep all three `uses:` lines on ONE
168+
# sha: the two verdicts must come from the same scanner.
141169
- name: Audit dependencies for known vulnerabilities (OSV-Scanner)
142-
uses: google/osv-scanner-action/osv-scanner-action@f4cfcc01edc9c8b756a9b873b7a623ca674da51e # v2.3.8
170+
if: github.event_name != 'pull_request'
171+
uses: google/osv-scanner-action/osv-scanner-action@f4cfcc01edc9c8b756a9b873b7a623ca674da51e # untagged, v2.5.0 image
143172
with:
144173
scan-args: |-
145174
--lockfile=pnpm-lock.yaml
146175
176+
# The checkout is the pull request's test merge (refs/pull/N/merge) at
177+
# depth 1. Its first parent is the base-branch commit GitHub merged the
178+
# pull request into -- the merge base of the scanned tree and the base
179+
# branch -- so ONE depth-1 fetch of that sha is all the history this needs
180+
# (never fetch-depth: 0). When neither pnpm-lock.yaml nor osv-scanner.toml
181+
# differs from it, the pull request cannot introduce an advisory and no
182+
# scan runs. The ledger is part of the condition: a pull request that
183+
# deletes an exemption changes the verdict without touching the lockfile.
184+
- name: Stage the OSV comparison against the merge base (pull request)
185+
id: osv-stage
186+
if: github.event_name == 'pull_request'
187+
run: |
188+
node scripts/osv-base-relative.mjs --self-test
189+
mapfile -t parents < <(git cat-file -p HEAD | sed -n 's/^parent //p')
190+
if [ "${#parents[@]}" -ne 2 ]; then
191+
echo "::error title=OSV base-relative verdict unavailable::HEAD is not the pull request's two-parent test merge (${#parents[@]} parent(s) found), so there is no merge base to judge against"
192+
exit 1
193+
fi
194+
base="${parents[0]}"
195+
git fetch --no-tags --depth=1 origin "$base"
196+
echo "base=$base" >> "$GITHUB_OUTPUT"
197+
if git diff --quiet "$base" HEAD -- pnpm-lock.yaml osv-scanner.toml; then
198+
echo "::notice title=OSV-Scanner verdict (base-relative)::lockfile unchanged — inherits main's verdict. pnpm-lock.yaml and osv-scanner.toml are identical to the merge base ${base:0:12}, so this pull request introduces no advisory; main's daily scheduled scan judges that lockfile."
199+
echo "compare=false" >> "$GITHUB_OUTPUT"
200+
exit 0
201+
fi
202+
for side in base head; do
203+
if [ "$side" = base ]; then rev="$base"; else rev=HEAD; fi
204+
mkdir -p ".osv-compare/$side"
205+
git show "$rev:pnpm-lock.yaml" > ".osv-compare/$side/pnpm-lock.yaml"
206+
git show "$rev:osv-scanner.toml" > ".osv-compare/$side/osv-scanner.toml"
207+
done
208+
echo "compare=true" >> "$GITHUB_OUTPUT"
209+
210+
# Both scans may exit non-zero (findings), so neither stops the job; the
211+
# judge below reads each result file against its step's outcome.
212+
# `--config` is explicit because the scanner looks for osv-scanner.toml
213+
# only in the scanned lockfile's own directory (measured on 2.5.0: a
214+
# ledger one directory up is not applied).
215+
- name: OSV-Scanner on the merge base's lockfile (pull request)
216+
id: osv-scan-base
217+
if: steps.osv-stage.outputs.compare == 'true'
218+
continue-on-error: true
219+
uses: google/osv-scanner-action/osv-scanner-action@f4cfcc01edc9c8b756a9b873b7a623ca674da51e # untagged, v2.5.0 image
220+
with:
221+
scan-args: |-
222+
--config=.osv-compare/base/osv-scanner.toml
223+
--format=json
224+
--output-file=.osv-compare/base/results.json
225+
--lockfile=.osv-compare/base/pnpm-lock.yaml
226+
227+
- name: OSV-Scanner on the pull request's lockfile (pull request)
228+
id: osv-scan-head
229+
if: steps.osv-stage.outputs.compare == 'true'
230+
continue-on-error: true
231+
uses: google/osv-scanner-action/osv-scanner-action@f4cfcc01edc9c8b756a9b873b7a623ca674da51e # untagged, v2.5.0 image
232+
with:
233+
scan-args: |-
234+
--config=.osv-compare/head/osv-scanner.toml
235+
--format=json
236+
--output-file=.osv-compare/head/results.json
237+
--lockfile=.osv-compare/head/pnpm-lock.yaml
238+
239+
- name: Judge the pull request's OSV findings against the merge base's (pull request)
240+
if: steps.osv-stage.outputs.compare == 'true'
241+
env:
242+
BASE_OUTCOME: ${{ steps.osv-scan-base.outcome }}
243+
HEAD_OUTCOME: ${{ steps.osv-scan-head.outcome }}
244+
BASE_SHA: ${{ steps.osv-stage.outputs.base }}
245+
# Read-only use: the open-issue search that names an inherited
246+
# advisory's finding card. The verdict never depends on it.
247+
GITHUB_TOKEN: ${{ github.token }}
248+
run: |
249+
node scripts/osv-base-relative.mjs \
250+
--base-results .osv-compare/base/results.json --base-outcome "$BASE_OUTCOME" \
251+
--head-results .osv-compare/head/results.json --head-outcome "$HEAD_OUTCOME" \
252+
--base-sha "$BASE_SHA"
253+
147254
- name: List outdated packages
148255
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
149256
run: |

0 commit comments

Comments
 (0)