A per-database collector that fails in SOME databases reports SUCCESS with no note #2623

Found while investigating why pg_column_stats and pg_index_bloat returned zero rows against a self-hosted PostgreSQL with 40 tables and 160 indexes.

What happens

The per-database fan-out — DarlingCollectorRunner.cs:720 and its Lite twin RemoteCollectorService.DefinitionRunner.cs:488 — escalates only when every database failed:

if (attempted > 0 && failed == attempted && firstFailure is not null)
    ExceptionDispatchInfo.Capture(firstFailure).Throw();

A partial failure increments failed, logs at debug, and falls through. The cycle then records SUCCESS, whatever rows the surviving databases produced, and a collection_note composed only from probe failures — which says nothing about a database that threw.

Why that is worse than it sounds

The three collectors broken by #2622's arity bug were all per-database. All three failed identically in the one database that had data. Only one of them — pg_extension_availability — surfaced as an ERROR, and only because it happens to return rows in every database including postgres, so all-failed tripped the escalation.

The other two had a second database (postgres) with legitimately nothing to report. It succeeded, failed (1) < attempted (2), and the cycle logged:

pg_column_stats | SUCCESS | 0
pg_index_bloat  | SUCCESS | 0

Zero rows, green, no note. Indistinguishable from a target that genuinely has no large tables — which is exactly what I assumed for most of an hour.

Scale it: a target with 30 databases where one is broken reports 29/30 of the truth as an unqualified SUCCESS, forever. Tolerating the failure is right — one unreachable database must not cost the other 29. Tolerating it silently is not.

Fix

Compose a note when failed > 0 && failed < attempted, naming the count, a capped list of the databases, and the first exception's message, and merge it with the probe-failure note rather than replacing it. Same change in both runners — this is a shared-seam defect, and the two have already drifted apart once on this exact loop.

Guard: a test that runs a definition throwing for one database of two and asserts the result carries a note naming the failed database. Without it the note is one refactor from being dropped, and its absence is silent by construction.