Skip to content

PostgreSQL targets enter the analysis pipeline: the pass routes by registry engine_kind inside the service, a pg_database_stats coverage witness replaces the tombstone, and every shared switch gains its one pg_ arm so the content lanes can build in parallel (#3542 v1 plumbing) - #3665

Merged
erikdarlingdata merged 2 commits into
devfrom
feat/3542-pg-analysis-plumbing
Sep 18, 2026

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

What does this PR do?

Headline mechanism. A PostgreSQL target now takes the scheduled analysis pass. DarlingAnalysisService holds two component sets — the SQL Server one it always composed, and a new PostgreSQL-target one (PgTargetFactCollector, PgTargetAnomalyDetector, PgTargetRelationshipGraph, PgTargetDrillDownCollector, PgTargetBaselineProvider, and a data-span gate over pg_database_stats) — and picks one per call by reading the registry's servers.engine_kind for the server being analyzed. The worker's two engine gates, their once-latch and the PostgresAnalysisNotApplicable tombstone are deleted. Every shared switch in PerformanceMonitor.Analysis gains exactly one delegating pg_ arm, so the nine content lanes that follow (#3542 v1 steps 2–9) each fill their own partial files and never touch a shared file again. This is the plumbing keystone of #3542 — partial, do not close.

The lie it stops telling

Until now a PostgreSQL target's analysis_state carried a tombstone: "Scheduled analysis does not apply to a PostgreSQL target … use the get_pg_* reads." That was honest when it was written (#2213/#2230): the pipeline read wait_stats, query_stats, cpu_utilization_stats, none of which a PostgreSQL server_id ever writes, so an ungated pass would have sat at "0 hours, still collecting" for the life of the deployment. It is wrong the moment a PostgreSQL-target engine exists, and the design (#3542, pg-analysis-counterpart-design.md) called for exactly that engine. After this PR, analyze_server against a PostgreSQL target with a day of pg_database_stats returns status = "empty" with a coverage block at 1.0 — a real pass over the right series with nothing yet to say — and the first scheduled pass clears the tombstone row (insufficient_data = false, message = NULL) with no migration (D7: the lazy overwrite).

Three corrections to the design doc, built to here

  1. Per-call resolution, not per-constructor (D4). The doc said "one service taking an engine — the constructor composes collector/detector/graph". It cannot: two of the service's three construction sites are process-wide singletons shared by every server (DarlingMcpHostService AddSingleton(new DarlingAnalysisService(...)), DarlingWebEndpoints), and the MCP tools resolve a server by NAME and never see an engine. So ResolveEngineAsync(serverId) reads engine_kind at the top of all three entry points (AnalyzeAsync, CollectAndScoreFactsAsync, ComparePeriodsAsync), uncached — engine_kind is written on both connect arms precisely so a re-pointed registration corrects it — and uncaught: a registry that cannot answer fails the pass through the pass's own classifier rather than defaulting a PostgreSQL target onto SQL Server tables. MonitoredEngineKind.IsPostgres (postgres / aurora-postgres) selects the PostgreSQL set; anything else, including NULL and an unknown token, selects the SQL Server set — a NULL makes no claim (PostgreSQL targets have no UI on either SKU, and both UIs render SQL Server tabs at them #2530), and that is today's behaviour for an unstamped row byte for byte. The one addition for the NULL case: when an unstamped row lands on the span gate, the insufficient-data message names the missing engine stamp and says the next connect fixes it, instead of letting "0 hours" read as "still collecting".
  2. A coverage witness on pg_database_stats. Analysis facts divide by the time the collector actually observed, and a window with a hole in it says so, so collector downtime no longer reads as improvement (#3538 A2) #3592 made the service's empty-window rule facts.Count == 0 || context.ObservedDurationMs <= 0. A collector that never stamps coverage makes EVERY pass unavailable ("the collector observed none of the window") — on a target whose collectors are running fine. PgTargetFactCollector.CollectObservedCoverageAsync runs first, over pg_database_stats (D3: the one universal one-minute series every flavour writes, no extension), and finishes through PgFactCollector.BuildCoverage, made internal static rather than copied. The SQL Server witness's text with one substitution — pinned as a transformation, so a rule added to one witness and not the other fails.
  3. PostgreSQL twins for Extreme, corroborated anomaly findings can now cross the notify floor, so the baseline engine is no longer notification-inert at shipped settings (Fixes #3526) #3584's SQL-Server-keyed anomaly registries. FactScorer.IsDeviationScoredAnomalyKey was a literal list of seven SQL Server families; IsExtremeAnomaly and ScoreAnomalyFact route through it, and IsTuningClassKey catches every ANOMALY_ prefix — so an ANOMALY_PG_* fact would have scored 0 and never left the 1.49 cap: the notification inertness Extreme, corroborated anomaly findings can now cross the notify floor, so the baseline engine is no longer notification-inert at shipped settings (Fixes #3526) #3584 fixed for SQL Server, shipping again for PostgreSQL on day one. PgTargetScorer.IsDeviationScoredAnomalyKey / IsPgRatioAnomalyKey register the PostgreSQL families by detector SHAPE (vocabulary, declared here); the shared ramp and the extremity escape then read the same AnomalyGate metadata for both engines. The reconciler consults PgTargetFactKeys.AnomalyToFamilies after its own map; the amplifier switch routes PgTargetScorer.IsPgKey ahead of the ANOMALY_ arm.

Why this shape

  • The whole D2 vocabulary is declared once, before any row persists (PgTargetFactKeys / PgTargetSources): eleven pg_-prefixed sources, every v1 key as a const, WaitKey(type, event) / BadActorKey(queryid) for the two dynamic families, ConfigAdvisoryRoots (D5: convention and posture checks only — never an evidence-gated or context key), AnomalyToFamilies. Keys persist in analysis_findings.root_fact_key and mute rules the moment the first row lands; this is the expensive-to-reverse decision made concrete. Content lanes reference constants, never literals.
  • Every source constant's name ends in Source so FactSourceRegistryTests' sweep (Source\s*=\s*"…") registers the PostgreSQL vocabulary the day it is declared — the shape the sweep's own doc blesses for WindowCoverage.FactSource. FactScorer.KnownSources is 15 → 26; both SKUs' get_analysis_facts descriptions spell the list out (the Lite literal is the one Lite edit, a census-pin boundary extension).
  • Stubs, one partial file per family, /* filled by lane N */. PgTargetScorer*.cs (0.0 / []), PgTargetAdvice*.cs (null), PgTargetRelationshipGraph*.cs (empty chains), PgTargetFactCollector.*.cs (return immediately), detector calls, baseline metric map. The dispatchers in each family's root file are the entire contract; a lane's edit never leaves its own file.
  • AllSql is reflection-derived (public const string *Sql) rather than hand-listed like PgFactCollector.AllSql, because eight lanes write this collector's SQL in parallel and a hand list is the merge conflict every one of them hits. The convention it rests on is pinned: every new NpgsqlCommand( in the family names such a const.
  • PgTargetBaselineProvider : PgBaselineProvider with one protected virtual ResolveBaselineQuery seam, inheriting the cache, the eight-column robust reader, RobustTierScaffold and the ONE IsCommandTimeout (BaselineTimeoutIsNamedTests) — not a second provider copying that machinery to swap one switch.
  • RelationshipGraph gains a protected (bool buildSqlServerEdges) ctor and protected AddEdge; the parameterless ctor is byte-identical (seven Lite tests construct it). PgTargetRelationshipGraph derives EMPTY — no SQL Server chain is reachable from a PostgreSQL fact (D2 made the keys disjoint), pinned.
  • One point-in-time fact the skeleton emits: PG_SERVER_MAJOR_VERSION (Value = major, metadata is_aurora, base 0) — the twin of SERVER_MAJOR_VERSION, read from the registry, emitted only when the major is known (never a fabricated 0). It is what lets a stub pass be empty rather than tripping the facts.Count == 0 half of the empty-window rule, and it is the fact the D6 flavour disclosure will hang on.
  • audit_config stops lying for a PostgreSQL target: it evaluated CTFP/MAXDOP/max-memory, found none, and said "no_config_data — the config collector may not have run yet" — false for a target whose pg_server_config collector runs hourly. It now answers not_collected naming the tool's scope and pointing at the CONFIG_PG_* facts; a projection into its recommendations shape is the knobs lane's content.
  • ToolRecommendations routes the three PostgreSQL prefixes to a pre-filled PgTargetToolRecommendations table (which get_pg_* read a family's table is a fact about the product, not a threshold), ahead of every SQL Server prefix arm.
  • Threshold lineage (analyze_server for PostgreSQL targets — the inference engine answers 'what is wrong, ranked' for one engine only #3605 ↔ Analysis-engine campaign: coverage honesty, measured thresholds, config attribution (adversarial review, 11 findings) #3538 A5/A7), executable before the first number exists: PgTargetThresholdLineageTests scans PgTargetScorer*.cs for a numeric bar (ApplyThresholdFormula(, a space-delimited comparison, Boost =) without a measured / engine-defined / unmeasured marker within six lines, and for any hour-in-milliseconds gate not keyed _per_hour. The scanner is exercised on arranged inputs so the rule is known to bite; it passes trivially on the stubs.

Rejected: (a) a sibling PgTargetAnalysisService — budget, abandon classification, persist, mute, notify would fork; (b) a shared Source = "waits" — the D2 cross-fire; (c) catching in ResolveEngineAsync and defaulting to SQL Server — persists a "0 hours" verdict for a PostgreSQL target on a transient fault; (d) writing the PG_CPU_PERCENT >= 80 load-family confirmer here — it is a bar, and bars are lane 9's under the lineage rule (the stub's comment names the arm).

Every shared-file arm, so nothing else moved

What it does NOT do

No schema (no rung, no StorageVersion bump). No content: no PostgreSQL SQL beyond the coverage witness, the span gate, the baseline-data gate and the registry metadata read; no threshold; no advice text; no graph edge; no baseline CTE. No Lite twin of the collector (D1 — Lite has no PostgreSQL target). No change to PgFactCollector* or PgFactCollectorTests (32-name Lite-parity census untouched). The same-statement-pileup PG gate in the worker stays — that one IS SQL-Server-shaped (query_snapshots). FactRemediation.Build* stays inert for PostgreSQL keys (D8: no Apply/DDL).

Blast radius

SQL Server pass: byte-identical, proven by running analyze_server (2 findings), get_analysis_facts and compare_analysis against the same planted store from origin/dev's binaries and this branch's, and cmp-ing the outputs (analysis_time normalised): identical, 11,923 bytes each. AnalysisCoverageLivePostgresTests, AnalyzeWindowEmptyLivePostgresTests, DarlingAnalysisPipelineTests, PgFactCollectorTests green on the live store. The only behaviour change reachable by a SQL Server target is the extra sentence on the insufficient-data message when — and only when — its registry row has NULL engine_kind.

Test plan

New (compile-verified here; the ungated ones and the live ones were also EXECUTED on this machine through a net10.0 harness against a throwaway timescale/timescaledb:2.28.1-pg18 store migrated to the current ladder — 367 of the 368 Darling.Tests in the harness green, the one red being the harness's own csproj tripping DocCommentHygieneTests' project census; first CI execution is the Windows job):

  • PgTargetFactCollectorTests — 12-name self-pin census + one-file-per-family + filled by lane N; AllSql closure (every const in the list, every NpgsqlCommand names a const); PG dialect pins; FROM/JOIN targets resolve to a collector table / servers / CTE and never a v_ view; coverage SQL = SQL Server witness with one substitution, finished through the shared BuildCoverage, first in emission order, no catch; deadline by reference + three-outcome degrade; metadata fact shape; gated e2e = the exit criterion (empty, coverage 1.0, 241 samples, one PG_SERVER_MAJOR_VERSION fact; get_analysis_facts source=pg_config; a 2-hour sibling held by the gate on the right series).
  • DarlingAnalysisServiceEngineRoutingTests — EngineFor over every MonitoredEngineKind token + NULL/unknown/case; two sets held for the service's life, distinct in every component; three entry points resolve before their first collector read, exactly three call sites, no cache field, no catch, CommandTimeout set; both span SQLs; gated e2e for the NULL-kind row (SQL Server set, message names the stamp; stamped → PostgreSQL series).
  • PgTargetSharedSwitchRoutingTests — vocabulary invariants; WaitKey normalisation (9 cases); behavioural: a PG z-score anomaly scores 1.0 / 0.5 / 0 through the shared ramp, the PG ratio anomaly takes the PG ramp (0 today — routing, since the SQL Server arm would read 1.0), SQL Server anomalies unaffected; a CONFIG_PG_* root roots a card at 0.4 on the PG graph while the evidence-gated key does not; PG graph empty for every SQL Server root, parameterless graph unchanged, seam accessibility; FactAdvice delegates every key and ANOMALY_PG_WAIT_PROFILE never reaches the ANOMALY_WAIT_ composer; reconciler folds ANOMALY_PG_DEADLOCK_RATE into PG_DEADLOCK_RATE same-db only.
  • PgTargetThresholdLineageTests — the rule, the A7 rule, and the scanner's own fixtures (bare bar / comparison / boost flagged; each marker shape accepted; trivial literals, prose, literal text ignored; six lines in, seven out).
  • PgTargetMcpSurfaceTests — every PG recommendation names a registered tool and only get_pg_* / get_analysis_*; every key has a row; story paths dedupe and never yield a SQL Server read; audit_config arm precedes the fact read; instructions name pg_database_stats; the HADR_SYNC_COMMIT rider.

Rewritten: CollectorRunnerConnectionEngineTests.TheScheduledAnalysisPassIsGatedByEngine → …RoutesByRegistryEngine (no tombstone/latch/engine test in the tick window; three ResolveEngineAsync call sites; no cache field). PostgresEngineGateBehaviorTests' two analyze_now cases → the PG target RUNS the pass (empty fixture: generic gate message, never "does not apply"; planted 25 h: Ran, analysis_state cleared) and the SQL Server target's message is byte-identical; reflective latch helper deleted, pass-capable fabricated worker added. FactSourceRegistryTests 15 → 26 + PgTargetSources.All equality. FactCollectorCommandTimeoutTests / FactCollectorFailureReportingTests globs widened to PgTargetFactCollector.*.cs.

Builds (zero warnings of ours): PerformanceMonitor.Analysis, PerformanceMonitor.Darling.Analysis, PerformanceMonitor.Darling.Service, Darling.Tests, Lite.Tests, deprecated/Dashboard.Tests (-p:EnableWindowsTargeting=true for the test projects). The one warning in the Lite.Tests build (LogTailOverlapThresholdPinTests.cs:95 xUnit2000) is pre-existing on dev from #3646.

Docs the change falsified, fixed: Darling/README.md (two sentences), docs/postgres-first-target-runbook.md ("what remains" bullet), DarlingMcpInstructions.cs (which series the 24 h is measured on, and that the pass routes by engine_kind).

Which component(s) does this affect?

  • Lite (one string literal: the get_analysis_facts source description, a census pin)
  • Darling
  • Lite Tests
  • Darling Tests
  • SQL collection scripts
  • Documentation
  • Full Dashboard (deprecated)
  • CLI Installer (deprecated)

How was this tested?

Above. Live store: throwaway TimescaleDB 2.28.1 / PostgreSQL 18 container on Docker context desktop-linux, migrated to the current ladder (V129 on dev after the rebase), torn down after. No production store or fleet server was read.

Checklist

  • I have read the contributing guide
  • My code builds with zero warnings (dotnet build -c Debug)
  • I have tested my changes against at least one SQL Server version (the SQL Server pass is exercised against planted SQL Server collector rows in the store; no live SQL Server was needed for a change that does not touch its reads)
  • I have not introduced any hardcoded credentials or server names

Partial for #3542 (v1 plumbing; steps 2–9 follow). Rider for #3653/#3616 (HADR_SYNC_COMMIT next_tools, Darling half of #3659's pair). CHANGELOG deliberately untouched — the coordinator splices this wave's entries.

…ng, a pg_database_stats coverage witness, and one pg_ arm per shared switch (#3542 v1 plumbing)
@erikdarlingdata
erikdarlingdata merged commit effe482 into dev Sep 18, 2026
5 of 7 checks passed
@erikdarlingdata
erikdarlingdata deleted the feat/3542-pg-analysis-plumbing branch September 18, 2026 23:45
@erikdarlingdata

Copy link
Copy Markdown
Owner Author

Coordinator review (review-dark window, in place of the bot): read the six FactScorer arms, ResolveEngineAsync (NULL/missing → SQL Server set via MonitoredEngineKind.IsPostgres, no inference — #2530), the DarlingWorker arm deletions (0 refs to the tombstone remain), the pg_database_stats coverage witness (DISTINCT collection_time — one row per database per collection — is the right correction to the plan), and the 55 stub markers the content lanes fill. SQL Server pass byte-identical by cmp. The PG_SERVER_MAJOR_VERSION metadata fact is the right call for plumbing over widening the shared empty-window rule tonight; the rule's "collection appears to have stopped" wording when a window IS observed but emits no fact is recorded on #3653 as a twin-SKU follow-up. Arming.

erikdarlingdata added a commit that referenced this pull request Sep 19, 2026
…a new capability — moved from Fixed to Added
erikdarlingdata added a commit that referenced this pull request Sep 19, 2026
…a new capability — moved from Fixed to Added (#3675)
erikdarlingdata added a commit that referenced this pull request Sep 19, 2026
…… NOT an all-clear" for a window the coverage witness proves the collector observed: the empty-window rule was `facts.Count == 0 || ObservedDurationMs <= 0`, written before any collector stamped coverage, and its first half made an observed window that simply produced no fact wear the dead-collector envelope — `unavailable` with a pointer at collection health on both analyze_server tools, the Viewer's dead-collector marker from the worker. The rule now gates on the witness alone; an observed window with no facts runs the pass and reads `empty` at its stated coverage, both SKUs (#3653) (#3687)

The dead-collector envelope (WindowEmptyMessage) is set on `ObservedDurationMs <= 0`
alone, both arms byte-identical to before. `facts.Count == 0` with observed time
falls through: WindowEmptyMessage stays null, the anomaly detector still runs, and
a zero-finding pass renders as the tools' existing `empty` shape with the coverage
block (#3665's PG exit criterion). One Information line names the coverage for the
scheduled pass. Partial coverage cannot reach that branch — the collector adds
COLLECTION_GAP whenever coverage is under the bar.

Tests: Lite DuckDB e2e arm (observed, zero facts → `empty`, coverage ≈ 1, no
dead-collector prose; fact count asserted through CollectAndScoreFactsAsync);
Darling live-PG twin with the captured Information line; a Darling.Tests
cross-SKU parity pin over the rule's shape and the composed message. Both
e2e arms proven red on the pre-change service (`Expected: "empty"`,
`Actual: "unavailable"`).
This was referenced Sep 19, 2026
erikdarlingdata added a commit that referenced this pull request Sep 20, 2026
…ption names its reader's two documented traps, the web catalogue's duration-trend line says rates and never percentiles, and server-tabs.js reads every Stamped tool by captured_at (partial #3653 A15/A16) (#3801)

The six drifts item 17 still listed were all delivered before this branch (#3665, #3696, #3697); three of the fixed texts had nothing holding them. Pins only, no wire change:

- DarlingMcpAgToolsSurfaceTests.Description_SurfacesTheReadersTwoDocumentedTraps anchors on DarlingAgReader's own bolded paragraph headers, then holds the description and the instructions paragraph to each trap's mechanism, consequence and tell (NO row for a server with no AGs / last non-empty snapshot / collection_time; lag and queue depth NOT banded / secondary_lag_seconds / is_suspended / 0 lag while suspended).
- DarlingMcpTrendToolsSurfaceAndSqlTests.QueryDurationTrend_CatalogueLine_NamesTheRatesThePayloadCarries_NotPercentiles holds the catalogue line to ms/sec + executions/sec + 'not percentiles', SerializeTrend to the two rates it projects, and the whole catalogue to 'percentile' only ever negated.
- ServerPageTabsTests.TheStampedReads_AreReadByTheirStamp_NeverTheRetiredKey derives the Stamped + SearchBound roster from McpLatestSnapshotStampTests.LatestTools, slices every stat/table/fanout call by bracket balance, and refuses key: "collection_time" in any descriptor array bound to one of them; the four named reads must be reached; SESSION_STATS must show captured_at.

Each red against its defect, then green, in the Mac harness.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant