Skip to content

fix(ci): the advisory-only vuln gate blocks CI when npm's advisory endpoint is down, despite documenting that it never blocks #686

Description

@mforce

Not a duplicate of #634 — that is a maintainability refactor of the same file whose body says explicitly "Nothing is broken". This is a live defect, in different lines, and it has failed CI on real PRs today.

What happens

.github/workflows/ci.yml runs the npm audit twice:

  • "Audit prod npm dependencies (high+, blocking)" — must fail closed. Correct today.
  • "Audit all npm dependencies (moderate+, advisory only)" — --warn-only, and the workflow comment says "ADVISORY only (never blocks)".

When registry.npmjs.org/-/npm/v1/security/advisories/bulk is unavailable, npm audit emits an error payload, and the advisory step fails the build:

npm warn audit 503 Service Unavailable - POST https://registry.npmjs.org/-/npm/v1/security/advisories/bulk
##[error][npm] unusable audit report — npm audit reported an error: {"summary":"","detail":""}
##[error]Process completed with exit code 2

Observed on PR #685 (run 33843411090), a PR that changes no web/ file at all.

Why

.github/scripts/vuln-gate.mjs has two bad-input paths, and neither consults warnOnly:

// ~line 358
console.error(`::error::[${options.ecosystem}] could not parse the audit output: ${err.message}`);
process.exitCode = 2; // a gate that cannot read its input must not pass silently

// ~line 366
console.error(`::error::[${options.ecosystem}] unusable audit report — ${shapeProblem}`);
process.exitCode = 2; // fail closed: an error payload is not "no vulnerabilities"

while the findings path does consult it:

// ~line 375
if (result.blocking.length > 0 && !options.warnOnly) process.exitCode = 1;

The file's own header documents the flag as --warn-only (report, always exit 0) — so the script contradicts its own contract, and ci.yml inherits the contradiction.

The distinction that matters

Fail-closed on missing data is correct for the blocking gate and must stay — "an error payload is not 'no vulnerabilities'" is the whole point of #146, and nothing here changes it.

For the advisory run it is incoherent: that step cannot block even when it finds a moderate advisory, so blocking when it finds nothing readable is strictly stranger. A step that never blocks on a finding should not block on an outage.

Fix

Make both bad-input paths respect --warn-only: emit ::warning:: and exit 0 when advisory, keep ::error:: and exit 2 otherwise. Add regression cases to .github/scripts/vuln-gate.test.mjs (already run by CI before the gate's verdict is trusted, so there is a natural home for them).

Cost so far

npm's advisory endpoint failed four CI runs today across two PRs and main. Roughly half were this avoidable path. Each one costs a full re-run and reads as a red cross on unrelated work.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions