Repository navigation
Commit dc759f2
ci(validate-deps): judge a pull request's OSV findings against the merge base; main keeps the absolute daily scan; job timeout 15 min (#22138)
Fixes #22082
Clause-②: no
## What this does
Implements ruling A as recorded on the card (comment 6049727506). On
`pull_request`, the OSV verdict of `validate-deps.yml` is now
base-relative. On `schedule` and `workflow_dispatch`, `main` keeps the
absolute verdict from the same step as before. That step's only change
is `if: github.event_name != 'pull_request'` (its version comment is
also corrected, see below). The job gains `timeout-minutes: 15`, sized
from the measured p50 of 1.3 min and max of 1.9 min.
- **Lockfile unchanged.** The new stage step compares the pull request's
test merge with its merge base. If neither `pnpm-lock.yaml` nor
`osv-scanner.toml` differs, the step prints the notice "lockfile
unchanged — inherits main's verdict" and runs no scan. The ledger counts
as part of the input because a PR that deletes an exemption changes the
verdict without touching the lockfile (replay R3 below).
- **Otherwise, both lockfiles are scanned.** The same pinned action
scans the PR's lockfile and the merge base's. Each is judged under its
own `osv-scanner.toml`, so the merge base's verdict is what `main`'s
scan gives and the PR's verdict is what the old absolute scan gave.
`scripts/osv-base-relative.mjs` then sorts the advisories:
- An advisory matched by both is **inherited**. It gets a `notice`
annotation that names it, the package versions each side matched, and
the open finding card that names it, when one exists. It does not fail
the step.
- An advisory matched only by the PR's lockfile is **introduced**. It
gets an `error` annotation and fails the step, as before.
- **The merge base.** `actions/checkout` puts the PR's test merge
(`refs/pull/N/merge`) at depth 1. Its first parent is the base-branch
commit that GitHub merged the PR into, which makes it the merge base of
the scanned tree. The step reads that parent from the commit object and
fetches it with one depth-1 `git fetch` (never `fetch-depth: 0`). If
HEAD is not a two-parent merge, the step fails with an error annotation.
- **Fail-closed.** A side is read only when its result file parses and
its step outcome matches its finding count (success means zero findings,
failure means some). In every other case the step fails and names that
side.
This adds no new gate, changes nothing that `main`'s scan judges, and
adds no exemption. `osv-scanner.toml`,
`scripts/check-osv-exemptions.mjs` and the other gate scripts are
untouched, and they still run first and unchanged.
## Why a helper script, and why not upstream's `osv-reporter-action`
The upstream reporter was read at `google/osv-scanner` v2.5.0
(`cmd/osv-reporter/main.go`,
`internal/ci/vulnerability_result_diff.go`). Three things rule it out:
- **It fails open.** When the new-side result file is missing or does
not parse, it logs a warning, treats the side as having no findings and
exits 0. Upstream's own reusable PR workflow runs both scans with
`continue-on-error: true`, so a scan error on the PR side passes the
gate.
- **It needs both sides at the same source path.** The diff keys on
`source.path`, so the base lockfile would have to be swapped into the
workspace and restored mid-job.
- **It cannot name the inherited advisories**, and the ruling requires
that notice.
The helper is one definition of introduced versus inherited, with the
scan-error guard built in. Its `--self-test` (23 cases, floor pinned)
runs as the first line of the stage step, so `check:self-test-wired`
sees it. The helper is in the `paths:` filter, so a PR that edits it
runs this workflow.
## Measured mechanism (osv-scanner 2.5.0, the binary in the pinned
action's image)
- **The pin.** `f4cfcc01edc9c8b756a9b873b7a623ca674da51e` is an untagged
upstream commit between the v2.5.0 and v2.5.1 tags. Its `action.yml`
runs `ghcr.io/google/osv-scanner-action:v2.5.0`, and run 37483796939's
job steps include "Pull ghcr.io/google/osv-scanner-action:v2.5.0". The
`# v2.3.8` comment had been stale since Dependabot's sha bump (#9209).
It is corrected, and all three `uses:` lines stay on that one sha.
- **The image's binary matches the release binary.** It is built from
the same `cmd/osv-scanner` with the same flags (goreleaser builds
`osv-scanner-action`). The release binary `osv-scanner_linux_amd64`
matched its published SHA256SUMS (`edcfc41d…`). The action's entrypoint
is upstream's `exit_code_redirect.sh`, which rewrites exit 128 (no
packages) to 0.
- **The ledger is found only in the lockfile's own directory.** With a
ledger one directory up, an ignored advisory was still reported. The
workflow therefore passes `--config` explicitly on both sides. This
settles dispatch assumption ②: without `--config`, a base lockfile
staged at another path is scanned with no ledger at all.
- **Exit codes, with `--config` and `--output-file`:**
| case | exit | result file |
|:--|:--|:--|
| no findings | 0 | written, `results: []` |
| findings | 1 | written |
| expired `ignoreUntil` | 1 | written; the advisory is reported again
(the ledger's expiry convention holds on both sides) |
| unknown key / broken TOML / missing `--config` file | 127 | **none** |
| missing lockfile | 127 | **none** |
| no packages | 128, rewritten to 0 by the action | **none** |
Because the outcome alone cannot tell findings from errors, the helper's
file-plus-outcome agreement check is what keeps the verdict closed.
## The pin
The body is written once, when the PR is created and before any run
exists, so the run ids are posted in the `os-dev-report` comment on
#22082, keyed by the head shas named here.
**Real runs** (seat amendment, from the dev's report on #22082). All are
`validate-deps.yml` on `pull_request`, against the merge base
`ef1fcb26a24f` (the `main` tip):
| Head | Run (job) | Result |
|---|---|---|
| `ce598b1ab7` | `37719042481` (`113122177520`) | success, with the
"lockfile unchanged" notice |
| `501569be49` (probe: `minimist` 1.2.5) | `37719190017`
(`113122655549`) | **failure**, one error: `GHSA-xvch-5gv4-984h`
(minimist@1.2.5) |
| `3576cdc334` (probe reverted) | `37719368398` (`113123226526`) |
success, with the "lockfile unchanged" notice |
| `5473cad514` (probe: `minimist` 1.2.8, no advisory, lockfile changed)
| `37719558844` (`113123838149`) | success, with **six inherited
notices** for `next@16.3.6` |
| `cc98faf47a` (final; same tree as `ce598b1ab7`) | `37719733046`
(`113124391631`) | success, with the "lockfile unchanged" notice |
`main` at `ef1fcb26a24f` carries those six advisories (filed as #22148).
So:
- the inherited pin is a **real run**: `37719558844`;
- the `scripts`-only pin is `37719368398` / `37719733046`, against that
advisory-carrying `main`;
- the replays R1–R3 below are supplementary.
### "A lockfile that introduces a new advisory fails": real run
- **Head `501569be49`** is a TEMPORARY probe commit. It adds root
devDependency `minimist` 1.2.5 (GHSA-xvch-5gv4-984h, fixed in 1.2.6)
through `pnpm add -D -w`. The PR squashes, and `3576cdc334` reverts the
probe.
- **Expected run result:** the stage step reports `compare=true`, both
scans run, and the judge fails with exactly one `error` annotation, for
GHSA-xvch-5gv4-984h (minimist@1.2.5).
- **Pre-check run locally**, with the same steps against main
`8fc50b7647` and the 2026-10-06 database snapshot described under R1–R3:
```
::error title=OSV advisory introduced by this pull request::GHSA-xvch-5gv4-984h (minimist@1.2.5) is matched by this pull request's lockfile and not by the merge base's (8fc50b7). ...
OSV base-relative verdict against the merge base (8fc50b7): 1 introduced, 0 inherited, 0 resolved.
judge exit=1
```
### "Lockfile unchanged": real runs
- The heads are `ce598b1ab7` (the implementation, at PR open) and
`3576cdc334` (after the probe's revert). The two trees are identical:
both are `d477febb20`.
- This PR changes neither `pnpm-lock.yaml` nor `osv-scanner.toml`, and
`.github/workflows/validate-deps.yml` is in the `paths:` filter, so the
workflow runs on it.
- **Expected:** a pass with the notice "lockfile unchanged — inherits
main's verdict", and the three scan and judge steps skipped.
- This branch runs no scan, so `main`'s advisory state cannot reach it.
That is why a `scripts`-only PR against a `main` that carries an
advisory passes here. R1 replays exactly that case.
### "Inherited passes with a notice": real run `37719558844` (above);
the replays below are supplementary
`main` is clean today: scheduled run 37565270565 succeeded. A real PR
run here cannot have a merge base that carries an advisory unless `main`
is touched or a throwaway branch is made, and both are out of bounds.
Following the dispatch, the step logic was replayed on real historical
pairs instead. The replay worked like this:
- It built the test merge GitHub would build (`git merge-tree
--write-tree BASE PRHEAD`, then `git commit-tree -p BASE -p PRHEAD`) in
an isolated clone.
- It ran the stage step's `run:` text verbatim, as extracted from this
branch's workflow. The only change was the helper's path, because the
historical trees carry no copy of the helper.
- It ran each scan step's `scan-args` verbatim through upstream's own
`exit_code_redirect.sh` and the release binary, with outcome = exit 0 ?
success : failure.
- It ran the judge step's `run:` text verbatim.
**The gap, named:** this container's egress policy refuses
`api.osv.dev`. The scans therefore ran `--offline` against a local
snapshot of the OSV npm database. That snapshot is an `all.zip` of
230,017 records, with max `modified` 2026-10-06T16:30:04Z and sha256
`0bd50081c140…`. It was found already on this container's disk, and its
provenance is otherwise unknown. Its GHSA-wq5f-xc86-pv6w (sharp,
published 13:43Z) and GHSA-pqg4-j6r4-53mv (shell-quote, published
13:40Z) records are those finding card #22013 measured. The anchor
lookup used the real repo-scoped listing.
**R1: PR #22002 as run 37483796939 saw it (the card's specimen: a
`scripts`-only change).** Base is `f0022c46c1`, `main`'s tip when the
run was created (15:00:57Z). PR head is `95c510eb8a`.
```
osv-base-relative self-test: 23 case(s), all held (floor 23)
::notice title=OSV-Scanner verdict (base-relative)::lockfile unchanged — inherits main's verdict. pnpm-lock.yaml and osv-scanner.toml are identical to the merge base f0022c4, so this pull request introduces no advisory; main's daily scheduled scan judges that lockfile.
GITHUB_OUTPUT: base=f0022c46c10a142f5fb29e6891a4fc06ec05a296 / compare=false
-- for contrast, the old absolute step on the same test merge's lockfile:
| https://osv.dev/GHSA-wq5f-xc86-pv6w | 8.9 | npm | sharp | 0.35.4 | 0.35.5 | pnpm-lock.yaml |
| https://osv.dev/GHSA-pqg4-j6r4-53mv | 9.2 | npm | shell-quote | 1.10.0 | 1.11.0 | pnpm-lock.yaml |
Exit code: 1
```
**R2: PR #21951 (a real lockfile-changing PR against a `main` that
carries advisories).** Base is `4c49150e0c`, PR head is `539b0752cd`.
The constructed merge's `pnpm-lock.yaml`, `osv-scanner.toml` and
`pnpm-workspace.yaml` blobs equal those of the PR's real squash commit
`787104baa9`.
```
GITHUB_OUTPUT: base=4c49150e0cb27a493390c4ac24e0b93e13efe662 / compare=true
osv-scan-base: found 1372 packages; GHSA-hp3w-g68c-fv3c filtered (the base's ledger); Exit code: 1 -> outcome failure
osv-scan-head: found 1371 packages; GHSA-hp3w-g68c-fv3c filtered (the PR's ledger); Exit code: 1 -> outcome failure
::notice title=OSV advisory inherited from the merge base::GHSA-pqg4-j6r4-53mv (here: shell-quote@1.10.0; at the merge base: shell-quote@1.10.0) is matched by the merge base's lockfile (4c49150) too, so it is main's to fix and does not fail this pull request; no open issue names it yet; main's daily scheduled scan is the signal that files its finding card.
::notice title=OSV advisory inherited from the merge base::GHSA-wq5f-xc86-pv6w (here: sharp@0.35.4; at the merge base: sharp@0.35.4) is matched by the merge base's lockfile (4c49150) too, so it is main's to fix and does not fail this pull request; no open issue names it yet; main's daily scheduled scan is the signal that files its finding card.
OSV base-relative verdict against the merge base (4c49150): 0 introduced, 2 inherited, 3 resolved.
resolved GHSA-238p-pmpm-9mq7 katex@0.16.47
resolved GHSA-68fv-2mgg-jv7q source-map-js@1.2.1
resolved GHSA-jqcg-44mw-7w3h proxy-addr@2.0.7
judge exit=0
-- the old absolute step on the same lockfile: sharp + shell-quote, Exit code: 1
```
The anchor clause reads "no open issue names it yet" because #22013 is
closed today. A live probe of the same listing found the open issues
that name a given advisory id. It also turned up a seat board post
quoting one, which is why board posts (`pm:seat`) are skipped; the
self-test pins both the anchor case and the board-post skip.
**R3: a synthetic ledger-only PR** that deletes the sprintf-js exemption
from `787104baa9`'s `osv-scanner.toml` and touches no lockfile:
```
GITHUB_OUTPUT: base=787104baa9ecca77ff13a8e83a4b7349728feadc / compare=true (no short-circuit: the ledger differs)
::error title=OSV advisory introduced by this pull request::GHSA-hp3w-g68c-fv3c (sprintf-js@1.1.3) is matched by this pull request's lockfile and not by the merge base's (787104b). ...
OSV base-relative verdict against the merge base (787104b): 1 introduced, 2 inherited, 0 resolved.
judge exit=1
```
## Helper ablations (each run with `scripts/ablation-replace.mjs`, which
restores the file and confirms the restored blob equals HEAD)
- **A1: `compare` ignores the base** (`if (base.has(id)) inherited.push`
becomes `if (false) inherited.push`). The self-test fails: 25 failed
expectations over 23 cases, exit 1.
- **A2: a missing result file read as "no findings"** (upstream's
fail-open). The self-test fails: 4 failed expectations, exit 1.
- **Restore:** the blob is back to `f35d31ee1072` = HEAD, and `git diff
HEAD` is empty.
## Local verification
These gates ran at `bf15ed3831`, plus `ce598b1ab7` for the entry-guard
fix. The final tree `d477febb20` is identical at `ce598b1ab7` and
`3576cdc334`.
- **Derived gates:** `node scripts/pm/dispatch-gates.mjs --commands`
derived 54 commands over the actual diff, and all 54 exit 0.
- **One red caught and fixed:** `check:entry-guard` first went red
because the helper exported bindings and ran on import. The fix puts the
dispatch behind `isEntrypoint`, and the gate is now green.
- **`pnpm check:pm-dispatch-gates`:** 1976 cases pass, in 820.5s.
- **`--ran` reconciliation:** 54 derived, 54 run, 0 NOT-MEASURED.
- **eslint on the new file:** `npx eslint --no-inline-config --format
json scripts/osv-base-relative.mjs` reported 1 file, 0 errors and 0
warnings.
- The file is in eslint's population: a JSON result came back for it,
not an "ignored" warning.
- The config enables no type-aware linting (`eslint.config.mjs` has no
`parserOptions.project`), so this diff cannot change the verdict on any
file it does not touch.
- The whole-repo `pnpm lint` belongs to CI.
## Acceptance notes
- **The unit is the advisory id,** following the ruling's wording ("an
advisory matched by both is reported as inherited"). Suppose a PR adds a
second vulnerable version of a package for an advisory `main` already
carries. That advisory reads as inherited, and the notice lists both
sides' package versions so the addition is visible. `main`'s daily scan
still judges it. Upstream's reporter counts occurrences instead.
- **Prose that drifts with this change, in files outside this PR's
allowed surface.** Both are out of bounds for this PR, and this PR has
no carrier for them:
- The `osv-scanner.toml` header says the workflow "blocks on any
advisory at any severity". That is still true of `main`'s scan, but a PR
now answers only for what it introduces.
- The `osv-scanner.toml` and `scripts/check-osv-exemptions.mjs` headers
cite measurements "against v2.3.8". The pinned image has been v2.5.0
since the Dependabot bump.
- **Remaining inherited-red path, by design:** an exemption that expires
on `main` still reddens every dependency-touching PR through the
unchanged "Verify OSV exemptions carry an expiry and a reason" step,
which the ruling keeps as is.
- **The job keeps `issues: write`,** which no step in it uses. The
judge's anchor lookup only reads. Permissions are unchanged here.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01VF48aw8RPG6wzDnMgp6rtw)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent d0bb78e commit dc759f2
2 files changed
Lines changed: 690 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
20 | 23 | | |
21 | 24 | | |
22 | 25 | | |
| |||
32 | 35 | | |
33 | 36 | | |
34 | 37 | | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
35 | 43 | | |
36 | 44 | | |
37 | 45 | | |
| |||
138 | 146 | | |
139 | 147 | | |
140 | 148 | | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
141 | 169 | | |
142 | | - | |
| 170 | + | |
| 171 | + | |
143 | 172 | | |
144 | 173 | | |
145 | 174 | | |
146 | 175 | | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
147 | 254 | | |
148 | 255 | | |
149 | 256 | | |
| |||
0 commit comments