The four PostgreSQL host alerts fire with the tier they earned instead of Severity: null, so their history rows stop wearing the colour their name implies (#3653 A8e, PostgreSQL half)

[erikdarlingdata]

What does this PR do?

The PostgreSQL half of #3653's A8e row: the four host alert arms in DarlingWorker — High CPU, Deadlocks Detected, Blocking Detected, Long-Running Query — fired with Severity: null, so once #3635 taught the history grids, get_alert_history and every channel to render the tier a row fired at, these four had nothing to render and fell back to the colour their NAME implies, while their SQL Server twins have carried the tier they fired at since #3660. Every PostgreSQL Deadlocks Detected row was red (one deadlock and a storm alike); every PostgreSQL High CPU row was amber (a 100%-of-ceiling fire indistinguishable from an 80% one while the fleet card beside it was red); Blocking and Long-Running Query rows were amber by implication rather than by statement. Partial for #3653 — ticks the "PG host arms still fire Severity: null" item; references #3653, does not close it.

Mechanism

DarlingAlertDeliverer.SendAndRecordAsync folds AlertOutcome.Severity into Context.SeverityOverride before the row is written (#2090), and since #3635 the serializer carries that as the row's Severity member — so a tier passed on the outcome at the fire site is the tier the grids read. These four arms passed null. Nothing downstream was wrong; the fire sites simply said nothing.

What changed, per arm — existing measured bars only, no new constants

High CPU → GradePgCpuFire(cpuPercent, acuUtilizationPercent), new, pure, internal static beside the arm. It is ServerHealthClassifier.CpuSeverity — the fleet card's classifier — called on the card's own two inputs (raw Performance Insights CPU + ACU utilization) on the PerformanceInsights arm, mapped Critical→Critical, everything else→Warning. Not a copy of the ladder: the classifier runs FleetCpuProvenance.CpuBandInputPercent itself to pick the capacity percent (#3281), so the figure graded is the figure the gate thresholded, by construction. Critical lands at ServerHealthThresholds.CpuCriticalPercent (95% of the configured ACU ceiling, never percent-of-allocated). The doc says what that bar is and is not, as #3660's GradeCpuFire does for SQL Server: the one ladder every card surface bands on (#3539 A2), stated against a quantity, not a fleet percentile — the claim is that the row and the card must not disagree about the same minute. Unknown (no capacity reading) cannot reach the fire site (breaching requires it) and maps to Warning rather than throwing.

Deadlocks Detected → AlertEngine.GradeDeadlockFire(count, rateTiers) — #3660's grader, the same count over the same window, against the same store-tunable pair. The PG arm's window is AlertEngine.RollingCountWindowHours (one hour), the constant the grader divides by, so the count IS the rate: Warning through 19/hr, Critical at deadlock_critical_per_hour (shipped 20/hr, #3368's measured tier). rateTiers is DarlingAlertSettings.DeadlockRateThresholds — the V120 knobs the PostgreSQL fleet card bands the same server on since #3638 — read once beside the count threshold (#3444's idiom), never DeadlockRateThresholds.Default, so an operator who raised the card's Critical to 31 sees the alert grade Warning at 25.

Why the alert's own count and not the card's counter. The brief asked me to reuse the differenced pg_stat_database.deadlocks rate #3638 computes for the fleet band. I did not, and the code says why: that rate lives in the fleet reader's own SQL (FleetPgDeadlockSql, run on demand for get_fleet_overview) and is not in hand in the alert pass — reusing it would be one more alert-path store read per PostgreSQL server per sweep for a number the row cannot display. The row quotes the deduped pg_deadlocks report count; grading it on a different number would be a second card/alert disagreement in place of the one this ends. The tiers are shared; the instrument is the alert's own, exactly as #3660's SQL Server arm grades its own graph count. One stated edge: the read caps at 50 distinct reports, so the grade saturates there — a Critical tier raised past 50/hr is unreachable on this host (shipped 20 is well under; the message quotes the same capped count, so grade and text agree). README states it.

Blocking Detected → PgBlockingFireSeverity = Warning; Long-Running Query → PgLongRunningQueryFireSeverity = Warning — explicit, and ONLY Warning, the #3660 tempdb shape. No measured bar exists for either and the constants' docs name what was considered: the SQL Server blocking band's tiers (#3596) were measured in reports-per-server-hour on engine-recorded evidence, while this alert counts distinct roots seen in a once-a-minute pg_stat_activity sample — the denominator mismatch that keeps the PG blocking band Unknown-with-reason; the sampled-shape distribution an honest tier needs has not been taken (#3653's #3539 residue names it, and #3601's lock_wait events as the source). LRQ has only the operator's threshold, a fire bar; "N× the threshold" would be a folklore multiplier, and the measured evidence that exists (the SQL Server twin's population, #3653 A5: 191 distinct ≥30-min sessions in 7 days, permanent background) argues a duration tier would grade the background as the emergency. What changes is that the row now says Warning; when a measured bar exists it lands as a second tier with nothing to unpick.

The tier rides the outcome only — Severity: grade — matching every graded fire already in this file (the Tier-0 trio and Poison Wait pass finding.Severity the same way). The SQL engine sets both context and outcome because it serves two SKUs' deliverers; this host serves one, whose fold is the documented contract. The CPU arm keeps Context: null (the deliverer creates one to hold the tier, as it does for every self-alert).

What it does NOT do

• Does not close Brains-review campaign: deferred structural residue (from #3538 / #3539 / #3540 / #3541) #3653 — one checklist item.
• Does not change when any of the four fires — only the tier the fire carries. No read changed, no LIMIT widened, no new knob, no migration, no CHANGELOG.md edit (entry handed to the coordinator).
• Does not touch the analysis-pass region of DarlingWorker (PostgreSQL-target analysis engine: design and v1 build #3542's lane), PgTarget*, DarlingAnalysisService, the PostgresAlertEvaluator's Tier-0 trio, AlertEngine.cs, AlertSeverity.ForMetric's by-name arms (pre-change PG rows keep the faithful replay: red for deadlocks, amber for the rest), or AlertMetricClassifier.
• Does not grade the SQL Server engine's Blocking Detected / Long-Running Query fire sites, which also still pass no tier (blockingContext?.SeverityOverride / lrqContext?.SeverityOverride, always null) — noted below as an out-of-lane finding for the coordinator, not this PR's file.
• AlertReadFailureSurfaceTests census: no catch block added or removed; no new await inside any try block.
• Two censuses fired on the first CI run, as designed, and were answered rather than dodged (second commit): FleetCardPostgresCpuTests' CPU-decider census now expects DarlingWorker.cs as the third file under Darling/ that calls ServerHealthClassifier.CpuSeverity( — a decider that reaches the shared decision is exactly what the census exists to force — while its caller census (TotalNonIdleCpuPercent + ClassifyCpuSource) stays at the two cards, with the comment saying why the alert cannot be one (its source is PerformanceInsights by construction; there is no second collector to classify). The test is renamed TheOnlySurfacesThatBandCpu_AllReachTheSharedDecision (one cref updated). RepoFileAdoptionTests' LF-reader census gains DarlingPgOperationalAlertTests.cs, because the fire-site pin anchors Severity: null, on the line break that follows it.

Blast radius

• Every new PostgreSQL fire of the four names persists "Severity":"Warning"|"Critical" in context_json; get_alert_history reports severity_source: fired for them. Visible colour changes: PG Deadlocks Detected rows at 1–19/hr go red→amber; PG High CPU rows at ≥95% of ceiling go amber→red. Blocking / LRQ stay amber (source metric_name→fired); the Blocking channel badge reads WARNING instead of the by-name ALERT.
• Three existing source pins slice EvaluatePgCpuAsync / EvaluatePgBlockingAsync "to the next private member" (PgCpuCapacityHeadroomTests, BuiltinAlertPersistenceRungTests, DarlingPgBlockingReaderTests); the new internal helpers are placed BEFORE their arm's doc comment so those slices still cover exactly one method. Verified by running the slicer verbatim (below).

Tests

Darling.Tests/DarlingPgOperationalAlertTests (the file that already holds these arms' pure seams), new section:

• GradePgCpuFire_IsTheCardsClassifierOnTheCapacityPercent_CriticalAtTheBandsBar — theory: (100,80)/(100,94.9)→Warning, (100,95)/(100,100)→Critical, and the raw CPU is NOT the input: (100,30)→Warning, (20,96)→Critical; each row cross-checked against ServerHealthClassifier.CpuSeverity.
• GradePgCpuFire_WithNoCapacityReading_IsWarning_NeverGradedOnTheRawCpu.
• PgDeadlockFire_IsTheSharedGrader_OnTheStoresTiers — 1/5/19→Warning, 20/50→Critical on Default; DarlingAlertSettings(config).DeadlockRateThresholds reads (7,31) → 25 Warning, 31 Critical; RollingCountWindowHours == 1.
• PgBlockingAndLongRunningQuery_FireAnExplicitWarning_AndNeverCritical.
• PgHostAlertArms_PassTheirGradeAtTheFireSite_NoneFiresSeverityNull — per-arm source slice: the expected Severity: <grade>, present, Severity: null,\n absent, DeadlockRateThresholds.Default absent from the deadlock arm; positive control that the exact code spelling still exists in AlertEngine.cs; the deliverer's context.SeverityOverride ??= outcome.Severity; fold pinned.
• APgHostFire_PersistsTheTierItFiredAt_SoTheRowRendersItInsteadOfTheName — drives the real DarlingAlertDeliverer (no channels configured, CapturingHistoryStore) with a deadlock-shaped fire (Incidents-only context, Warning) and a CPU-shaped fire (null context, Critical): AlertHistoryRowSeverity.Describe reads ("warning","fired") / ("critical","fired"), IsCritical flips against the by-name answer, the three incidents survive beside the tier, and a pre-Brains-review campaign: deferred structural residue (from #3538 / #3539 / #3540 / #3541) #3653 JSON of either name still reads metric_name.

Executed on this machine (both censuses included, the CPU one as a verbatim probe because its file references the WPF Viewer — 94 tests, 0 failed after the second commit) via the mactest pattern (net10.0 harness, AssemblyName=Darling.Tests, project-referencing the built Service + Storage): the whole class (30 tests), plus AlertMasterSwitchSurfaceTests, AlertReadFailureSurfaceTests, DarlingPgBlockingReaderTests and a verbatim copy of the two Viewer-referencing classes' EvaluatePgCpuBody() pins — 91 tests, 0 failed, 1 skipped (live PG). Mutation control: reverting the blocking arm to Severity: null fails the source pin (Assert.Contains() Failure: Sub-string not found).

Builds, 0 warnings, -c Release -p:EnableWindowsTargeting=true: PerformanceMonitor.Darling.Service, Darling.Tests (net10.0-windows). CI is the first execution under net10.0-windows.

Which component(s) does this affect?

• Lite
• Darling
• Lite Tests
• Darling Tests
• SQL collection scripts
• Documentation
• Full Dashboard (deprecated)
• CLI Installer (deprecated)

How was this tested?

See Tests. Pure grading of values the arms already had; the one integration seam (deliverer → persisted row → AlertHistoryRowSeverity) is driven end-to-end against the real deliverer. No SQL Server or PostgreSQL store was touched — no read changed.

Checklist

• I have read the contributing guide
• My code builds with zero warnings (dotnet build -c Debug)
• I have tested my changes against at least one SQL Server version (pure grading + real-deliverer drive; no engine read changed)
• I have not introduced any hardcoded credentials or server names

References #3653.