Repository navigation
ci(security): the npm audit gate fails an audit-endpoint error exactly like a high advisory — retry, and report 'could not check' distinctly #16337
Description
Activity
More evidence (from the merge-train reviewer session):
- The failing call is npm's
audits/quickendpoint. It answered400 Bad Request … Invalid package treewith the message "this endpoint is being retired". - The input didn't change. The frontend lockfile is byte-identical on base, security(auth): admin-gate the open routers, settings, teams and user-account routes (#15745, #16278, #16276, #16277, #15738, #16279) #16240 and the train (blob
2f2e5c35079e). With that exact lockfile, this job passed at 10:35 (run 34581977350), 10:43 (34581820886) and 11:37 (34587989756), then failed on the train at 12:07. That's the same input with a different result at a network call.
Acceptance criterion added:
- The gate no longer depends on the retiring
audits/quickendpoint. Identify which npm version and code path fall back to it, and move the gate onto the current bulk advisory endpoint, for example by pinning or upgrading npm, or by calling the audit the supported way. Prove it with a run whose log shows the bulk endpoint in use.
Why this matters: a retry only covers transient errors. Once the endpoint is retired, every frontend PR and base fails this job for good.
- The failing call is npm's
Design finding, from reading
.github/workflows/frontend-test.yml"Run npm audit". The step calls the audit service twice:npm audit --json > audit-results.json || truefetches the machine-readable report and uploads it as the artifact. On 2026-09-11 this call succeeded (1 low, 1 moderate).npm audit --audit-level=highis the gate, and it makes a second network call for data the first one already fetched. This is the call that got the 400.
Proposed fix, which also covers the ACs above:
- Gate on the JSON the step already has. Fail when
metadata.vulnerabilities.high + critical > 0, with the advisory message, unchanged. - If
audit-results.jsonis missing, can't be parsed, or carries anerrorobject, fail with a distinct "audit endpoint unavailable, could not check" message. That keeps the gate fail-closed. - Retry the single network call a bounded number of times, set by an env-backed constant, before concluding it's unavailable.
That's one network call instead of two, and "didn't look" is reported separately from "found". No
.npmrcis tracked in the repo, so nothing forces the legacy endpoint. Theaudits/quickhit appears to be npm's own fallback after its bulk call failed. Confirming that is part of the endpoint AC.AC2 amended. The review of #16357 asked for this change to be made explicit, rather than meeting the criterion in spirit.
- Before: "still fails with the advisory message, unchanged".
- Now: a high or critical finding still fails, labelled "advisories found", with the critical and high counts in the headline and the job summary.
- Why: the old message was npm's own
--audit-level=highoutput. The second audit call printed it, and the fix removes that call, because an endpoint failure on it read exactly like an advisory. - The full npm report is still uploaded as
audit-results.json.
- added a commit that references this issue
on Sep 12, 2026 Closing: all 4 criteria met on
Dev_new_gui(4a7e76e28), delivered by #16357.AC Evidence 1. Endpoint errors retried, persistent one fails naming the error pipeline-scripts/npm_audit_gate.py:101retry_delay_seconds;:208"Only unavailable is retried";ANNOTATION_TITLES[UNAVAILABLE] = "npm audit: could not check"(:63)2. High/critical still fails as "advisories found" with counts :229f"**Failed, advisories found:** {critical} critical and ..."3. Job summary states which of the three results happened ANNOTATION_TITLESmapsFOUND/UNAVAILABLEto distinct titles that land in the summary4. A fixture test covers all three npm_audit_gate_test.py:65(low/moderate pass),:81(endpoint error → unavailable, named),:122(no usable report → unavailable, never a pass),:163(timeout → unavailable)
Problem
On 2026-09-11, merge train #16321 went red on Frontend Testing Suite / Security Scan (run 34591254177, job 103246844388). The cause wasn't an advisory.
npm audit --audit-level=highexited 1 because the registry answerednpm error audit endpoint returned an error/error: 'Bad Request'. An earlier audit in the same job had already reported "2 vulnerabilities (1 low, 1 moderate)", both below the high gate.The gate can't tell "we couldn't check" apart from "we found a high advisory". Both are the same red check, and only reading the log separates them (
MEASUREMENT_DISCIPLINE.md: nothing found versus did not look). A registry hiccup therefore blocks a merge exactly the way a real finding does, and there's no retry.Proposal
npm auditJSON and error output.Related: #16131, where the same gate audits only 1 of 13 tracked
package.jsonfiles. The same workflow could take both.Acceptance criteria
--audit-level=highoutput, printed by the second audit call this fix removes, so it can't stay word-for-word the same. The full npm report is still uploaded as the artifact.