Add a repeatable script for reproducing customer CVE scans

scripts/scan-calamari-cves/README.md

@@ -0,0 +1,78 @@
+# Reproducing what customer scanners report against Calamari
+
+Run `./scan.sh`. Takes about ten minutes, most of it downloading a ~270 MB package.
+
+```bash
+./scan.sh                # latest main-branch CI build
+./scan.sh 2026.3.508     # a specific published version
+./scan.sh --local        # publish from your working tree and scan that
+```
+
+## Why this rather than `dotnet list package --vulnerable`
+
+**They answer different questions, and the local one over-reports.**
+
+Customers scan the files on their deployment targets. Calamari publishes
+**self-contained**, so a target holds ~345 DLLs including a full private copy of the .NET
+runtime. That is the scan surface.
+
+`dotnet list package --vulnerable` walks the NuGet *graph*, which includes build-time
+reference shims — `System.Net.Http 4.3.0`, `System.Text.RegularExpressions 4.3.0` — pulled
+in transitively via `NETStandard.Library`. Those contribute **no runtime assembly**
+(`runtime: []` in the shipped `.deps.json`, and the DLLs are absent from build output), so
+no customer scanner ever sees them. Dependabot doesn't report them either.
+
+If someone asks about a CVE that only appears in the local command, that's the explanation.
+
+## What the script does
+
+1. Downloads the real published `Octopus.Calamari.Consolidated` package from feedz.
+2. Extracts it, including the inner consolidated archive containing every flavour and RID.
+3. Prints the **bundled .NET runtime version** — usually the single most important number,
+   since a self-contained app ships its own runtime and inherits its CVEs.
+4. Scans with **Trivy** and **Grype** — two tools, two vulnerability databases. Customer
+   scanners disagree with each other, so one tool is not a baseline.
+5. Reports total matches and *distinct* CVEs. These differ a lot: the same finding repeats
+   across ~43 `deps.json` files, which is why a report of "42 vulnerabilities" can be one
+   issue.
+
+## Reading the results
+
+**Distinct count is the real number.** Total matches counts each flavour separately.
+
+**Check the runtime version first.** It's the largest single contributor. A self-contained
+app carries its own runtime, so an artifact built months ago carries a months-old runtime
+with every CVE published since.
+
+**Findings are version-dependent, and that's usually the answer.** Measured 2026-08-01:
+
+| Calamari | Distinct CVEs | Bundled .NET |
+|---|---|---|
+| `2025.3.417` | **9** (1 critical, 5 high) | 6.0.36 — **EOL Nov 2024** |
+| `2026.3.508` | **1** (medium) | 8.0.29 — current patch |
+
+If a customer reports many CVEs, check their Calamari version before anything else.
+
+**Beware the EOL trap.** The 2025.3.417 scan showed *zero* runtime CVEs despite bundling an
+end-of-life .NET 6. Microsoft stops publishing advisories for out-of-support versions, so
+scanners go quiet. **A clean runtime scan on an old artifact is not evidence it is safe** —
+it usually means nobody is looking any more.
+
+**Reported ≠ exploitable.** A finding against a shipped DLL says the version matches an
+advisory, not that the vulnerable path is reachable. Assessing that means reading how
+Calamari calls the library. Worked example: `CVE-2026-44788` (SharpCompress) is reported,
+but the vulnerable path is the archive-level `IArchive.WriteToDirectory()`, while Calamari
+iterates entries itself and calls the per-entry APIs the advisory names as guarded — and
+`ThrowIfPathTraversalAttempted` bounds-checks every entry key before any write. The
+regression test for it passes on the *vulnerable* version, which is the proof.
+
+## Caveats
+
+- Both tools key off `deps.json`. A scanner that fingerprints raw DLL file versions may
+  report differently.
+- `--local` uses *your* SDK's runtime pack, which may differ from CI's. A local publish
+  showed 8.0.27 with seven HIGH runtime CVEs while CI shipped 8.0.29 with none. Always
+  compare against a feed scan before reporting anything.
+- Vulnerability databases move daily. `CVE-2026-44788` was absent from the NuGet audit
+  source at 10:45 and present by 15:00 on the same day. Re-run rather than cite an old
+  result.

scripts/scan-calamari-cves/scan.sh

@@ -0,0 +1,120 @@
+#!/usr/bin/env bash
+# Reproduce what a customer's vulnerability scanner reports against Calamari.
+#
+# Customers scan the files on their deployment targets, not this repo and not the
+# NuGet graph. That distinction matters: `dotnet list package --vulnerable` reports
+# build-time reference shims that contribute no runtime assembly and that customer
+# scanners never see. This script scans the real shipped artifact instead.
+#
+# Usage:
+#   ./scan.sh                      # scan the latest main-branch CI build from feedz
+#   ./scan.sh 2026.3.508           # scan a specific published version
+#   ./scan.sh --local              # publish from the working tree and scan that
+#
+# Requires: docker (or OrbStack), curl, unzip, python3. dotnet only for --local.
+
+set -euo pipefail
+
+WORKDIR="${TMPDIR:-/tmp}/calamari-cve-scan"
+FEED="https://f.feedz.io/octopus-deploy/dependencies/nuget/v3"
+PKG="octopus.calamari.consolidated"
+MODE="feed"
+VERSION=""
+
+for arg in "$@"; do
+  case "$arg" in
+    --local) MODE="local" ;;
+    --help|-h) sed -n '2,15p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
+    *) VERSION="$arg" ;;
+  esac
+done
+
+say() { printf '\n\033[1m%s\033[0m\n' "$*"; }
+note() { printf '  %s\n' "$*"; }
+
+rm -rf "$WORKDIR"; mkdir -p "$WORKDIR/scan"
+
+if [ "$MODE" = "local" ]; then
+  say "Publishing from the working tree (self-contained linux-x64, as shipped)"
+  REPO_ROOT="$(cd "$(dirname "$0")/../.." && pwd)"
+  dotnet publish "$REPO_ROOT/source/Calamari/Calamari.csproj" \
+    -c Release -f net8.0 -r linux-x64 --self-contained true \
+    -o "$WORKDIR/scan" >/dev/null
+  note "published $(find "$WORKDIR/scan" -name '*.dll' | wc -l | tr -d ' ') DLLs"
+  note "NOTE: this uses YOUR SDK's runtime pack, which may differ from CI's."
+  note "      Compare against a feed scan before drawing conclusions."
+else
+  if [ -z "$VERSION" ]; then
+    say "Finding the latest published Calamari"
+    VERSION=$(curl -fsS "$FEED/registration/$PKG/index.json" \
+      | python3 -c "import json,sys,urllib.request
+d=json.load(sys.stdin); last=d['items'][-1]
+items=last.get('items')
+if items is None:
+    with urllib.request.urlopen(last['@id']) as r: items=json.loads(r.read()).get('items',[])
+vs=[(i.get('catalogEntry') or {}).get('version','') for i in items]
+vs=[v for v in vs if v and '-' not in v]
+print(vs[-1] if vs else '')")
+    [ -z "$VERSION" ] && { echo "could not determine latest version" >&2; exit 1; }
+  fi
+  note "version: $VERSION"
+
+  say "Downloading the shipped package (this is what customers receive)"
+  curl -fsS --max-time 600 -o "$WORKDIR/cal.nupkg" \
+    "$FEED/packages/$PKG/$VERSION/$PKG.$VERSION.nupkg"
+  note "$(du -h "$WORKDIR/cal.nupkg" | awk '{print $1}')"
+
+  say "Extracting"
+  mkdir -p "$WORKDIR/pkg"
+  ( cd "$WORKDIR/pkg" && unzip -qq ../cal.nupkg )
+  INNER=$(find "$WORKDIR/pkg/contentFiles" -name '*.zip' | head -1)
+  [ -z "$INNER" ] && { echo "no inner payload found" >&2; exit 1; }
+  unzip -qq "$INNER" -d "$WORKDIR/scan"
+  note "$(find "$WORKDIR/scan" -type f | wc -l | tr -d ' ') files"
+fi
+
+say "Bundled .NET runtime (self-contained, so this ships to every target)"
+# -I skips binaries; the deps.json files carry the authoritative version and the
+# self-contained helper binaries would otherwise emit "Binary file ... matches" noise.
+grep -rhoIE 'runtimepack\.Microsoft\.NETCore\.App\.Runtime\.[a-z0-9-]+/[0-9.]+' "$WORKDIR/scan" 2>/dev/null \
+  | sort -u | sed 's/^/  /' || note "none found"
+
+say "Trivy"
+docker run --rm -v "$WORKDIR/scan":/scan:ro -v "$WORKDIR/trivy-cache":/root/.cache \
+  --platform linux/amd64 aquasec/trivy:latest fs --scanners vuln --format json --quiet /scan \
+  > "$WORKDIR/trivy.json" 2>/dev/null || true
+python3 - "$WORKDIR/trivy.json" <<'PY'
+import json,sys
+try: d=json.load(open(sys.argv[1]))
+except Exception: print("  (no output)"); raise SystemExit
+seen=set(); total=0
+for r in d.get('Results') or []:
+    for v in (r.get('Vulnerabilities') or []):
+        total+=1
+        seen.add((v.get('Severity'),v.get('VulnerabilityID'),v.get('PkgName'),v.get('InstalledVersion'),v.get('FixedVersion')))
+print(f"  {total} matches across all flavours -> {len(seen)} DISTINCT")
+for s,c,p,i,f in sorted(seen):
+    print(f"  {s:9s} {c:22s} {p:38s} {i:14s} fixed: {f}")
+PY
+
+say "Grype (second opinion, different vulnerability database)"
+docker run --rm -v "$WORKDIR/scan":/scan:ro -v "$WORKDIR/grype-cache":/root/.cache \
+  --platform linux/amd64 anchore/grype:latest dir:/scan -o json -q \
+  > "$WORKDIR/grype.json" 2>/dev/null || true
+python3 - "$WORKDIR/grype.json" <<'PY'
+import json,sys
+try: d=json.load(open(sys.argv[1]))
+except Exception: print("  (no output)"); raise SystemExit
+seen=set()
+for m in d.get('matches') or []:
+    v=m.get('vulnerability') or {}; a=m.get('artifact') or {}
+    seen.add((str(v.get('severity')),str(v.get('id')),str(a.get('name')),str(a.get('version'))))
+print(f"  {len(d.get('matches') or [])} matches -> {len(seen)} DISTINCT")
+for s,c,p,ver in sorted(seen):
+    print(f"  {s:9s} {c:22s} {p:38s} {ver}")
+PY
+
+say "Done"
+note "artifacts kept in $WORKDIR (trivy.json / grype.json for raw detail)"
+note "If the two scanners disagree, prefer investigating over dismissing —"
+note "they use different databases and different matching rules."