Repository navigation
PostgreSQL targets enter the analysis pipeline: the pass routes by registry engine_kind inside the service, a pg_database_stats coverage witness replaces the tombstone, and every shared switch gains its one pg_ arm so the content lanes can build in parallel (#3542 v1 plumbing) - #3665
Conversation
…ng, a pg_database_stats coverage witness, and one pg_ arm per shared switch (#3542 v1 plumbing)
|
Coordinator review (review-dark window, in place of the bot): read the six |
…a new capability — moved from Fixed to Added
…a new capability — moved from Fixed to Added (#3675)
…… NOT an all-clear" for a window the coverage witness proves the collector observed: the empty-window rule was `facts.Count == 0 || ObservedDurationMs <= 0`, written before any collector stamped coverage, and its first half made an observed window that simply produced no fact wear the dead-collector envelope — `unavailable` with a pointer at collection health on both analyze_server tools, the Viewer's dead-collector marker from the worker. The rule now gates on the witness alone; an observed window with no facts runs the pass and reads `empty` at its stated coverage, both SKUs (#3653) (#3687) The dead-collector envelope (WindowEmptyMessage) is set on `ObservedDurationMs <= 0` alone, both arms byte-identical to before. `facts.Count == 0` with observed time falls through: WindowEmptyMessage stays null, the anomaly detector still runs, and a zero-finding pass renders as the tools' existing `empty` shape with the coverage block (#3665's PG exit criterion). One Information line names the coverage for the scheduled pass. Partial coverage cannot reach that branch — the collector adds COLLECTION_GAP whenever coverage is under the bar. Tests: Lite DuckDB e2e arm (observed, zero facts → `empty`, coverage ≈ 1, no dead-collector prose; fact count asserted through CollectAndScoreFactsAsync); Darling live-PG twin with the captured Information line; a Darling.Tests cross-SKU parity pin over the rule's shape and the composed message. Both e2e arms proven red on the pre-change service (`Expected: "empty"`, `Actual: "unavailable"`).
…ption names its reader's two documented traps, the web catalogue's duration-trend line says rates and never percentiles, and server-tabs.js reads every Stamped tool by captured_at (partial #3653 A15/A16) (#3801) The six drifts item 17 still listed were all delivered before this branch (#3665, #3696, #3697); three of the fixed texts had nothing holding them. Pins only, no wire change: - DarlingMcpAgToolsSurfaceTests.Description_SurfacesTheReadersTwoDocumentedTraps anchors on DarlingAgReader's own bolded paragraph headers, then holds the description and the instructions paragraph to each trap's mechanism, consequence and tell (NO row for a server with no AGs / last non-empty snapshot / collection_time; lag and queue depth NOT banded / secondary_lag_seconds / is_suspended / 0 lag while suspended). - DarlingMcpTrendToolsSurfaceAndSqlTests.QueryDurationTrend_CatalogueLine_NamesTheRatesThePayloadCarries_NotPercentiles holds the catalogue line to ms/sec + executions/sec + 'not percentiles', SerializeTrend to the two rates it projects, and the whole catalogue to 'percentile' only ever negated. - ServerPageTabsTests.TheStampedReads_AreReadByTheirStamp_NeverTheRetiredKey derives the Stamped + SearchBound roster from McpLatestSnapshotStampTests.LatestTools, slices every stat/table/fanout call by bracket balance, and refuses key: "collection_time" in any descriptor array bound to one of them; the four named reads must be reached; SESSION_STATS must show captured_at. Each red against its defect, then green, in the Mac harness.
What does this PR do?
Headline mechanism. A PostgreSQL target now takes the scheduled analysis pass.
DarlingAnalysisServiceholds two component sets — the SQL Server one it always composed, and a new PostgreSQL-target one (PgTargetFactCollector,PgTargetAnomalyDetector,PgTargetRelationshipGraph,PgTargetDrillDownCollector,PgTargetBaselineProvider, and a data-span gate overpg_database_stats) — and picks one per call by reading the registry'sservers.engine_kindfor the server being analyzed. The worker's two engine gates, their once-latch and thePostgresAnalysisNotApplicabletombstone are deleted. Every shared switch inPerformanceMonitor.Analysisgains exactly one delegatingpg_arm, so the nine content lanes that follow (#3542 v1 steps 2–9) each fill their own partial files and never touch a shared file again. This is the plumbing keystone of #3542 — partial, do not close.The lie it stops telling
Until now a PostgreSQL target's
analysis_statecarried a tombstone: "Scheduled analysis does not apply to a PostgreSQL target … use the get_pg_* reads." That was honest when it was written (#2213/#2230): the pipeline readwait_stats,query_stats,cpu_utilization_stats, none of which a PostgreSQL server_id ever writes, so an ungated pass would have sat at "0 hours, still collecting" for the life of the deployment. It is wrong the moment a PostgreSQL-target engine exists, and the design (#3542,pg-analysis-counterpart-design.md) called for exactly that engine. After this PR,analyze_serveragainst a PostgreSQL target with a day ofpg_database_statsreturnsstatus = "empty"with a coverage block at 1.0 — a real pass over the right series with nothing yet to say — and the first scheduled pass clears the tombstone row (insufficient_data = false, message = NULL) with no migration (D7: the lazy overwrite).Three corrections to the design doc, built to here
DarlingMcpHostServiceAddSingleton(new DarlingAnalysisService(...)),DarlingWebEndpoints), and the MCP tools resolve a server by NAME and never see an engine. SoResolveEngineAsync(serverId)readsengine_kindat the top of all three entry points (AnalyzeAsync,CollectAndScoreFactsAsync,ComparePeriodsAsync), uncached —engine_kindis written on both connect arms precisely so a re-pointed registration corrects it — and uncaught: a registry that cannot answer fails the pass through the pass's own classifier rather than defaulting a PostgreSQL target onto SQL Server tables.MonitoredEngineKind.IsPostgres(postgres/aurora-postgres) selects the PostgreSQL set; anything else, including NULL and an unknown token, selects the SQL Server set — a NULL makes no claim (PostgreSQL targets have no UI on either SKU, and both UIs render SQL Server tabs at them #2530), and that is today's behaviour for an unstamped row byte for byte. The one addition for the NULL case: when an unstamped row lands on the span gate, the insufficient-data message names the missing engine stamp and says the next connect fixes it, instead of letting "0 hours" read as "still collecting".pg_database_stats. Analysis facts divide by the time the collector actually observed, and a window with a hole in it says so, so collector downtime no longer reads as improvement (#3538 A2) #3592 made the service's empty-window rulefacts.Count == 0 || context.ObservedDurationMs <= 0. A collector that never stamps coverage makes EVERY passunavailable("the collector observed none of the window") — on a target whose collectors are running fine.PgTargetFactCollector.CollectObservedCoverageAsyncruns first, overpg_database_stats(D3: the one universal one-minute series every flavour writes, no extension), and finishes throughPgFactCollector.BuildCoverage, madeinternal staticrather than copied. The SQL Server witness's text with one substitution — pinned as a transformation, so a rule added to one witness and not the other fails.FactScorer.IsDeviationScoredAnomalyKeywas a literal list of seven SQL Server families;IsExtremeAnomalyandScoreAnomalyFactroute through it, andIsTuningClassKeycatches everyANOMALY_prefix — so anANOMALY_PG_*fact would have scored 0 and never left the 1.49 cap: the notification inertness Extreme, corroborated anomaly findings can now cross the notify floor, so the baseline engine is no longer notification-inert at shipped settings (Fixes #3526) #3584 fixed for SQL Server, shipping again for PostgreSQL on day one.PgTargetScorer.IsDeviationScoredAnomalyKey/IsPgRatioAnomalyKeyregister the PostgreSQL families by detector SHAPE (vocabulary, declared here); the shared ramp and the extremity escape then read the sameAnomalyGatemetadata for both engines. The reconciler consultsPgTargetFactKeys.AnomalyToFamiliesafter its own map; the amplifier switch routesPgTargetScorer.IsPgKeyahead of theANOMALY_arm.Why this shape
PgTargetFactKeys/PgTargetSources): elevenpg_-prefixed sources, every v1 key as aconst,WaitKey(type, event)/BadActorKey(queryid)for the two dynamic families,ConfigAdvisoryRoots(D5: convention and posture checks only — never an evidence-gated or context key),AnomalyToFamilies. Keys persist inanalysis_findings.root_fact_keyand mute rules the moment the first row lands; this is the expensive-to-reverse decision made concrete. Content lanes reference constants, never literals.SourcesoFactSourceRegistryTests' sweep (Source\s*=\s*"…") registers the PostgreSQL vocabulary the day it is declared — the shape the sweep's own doc blesses forWindowCoverage.FactSource.FactScorer.KnownSourcesis 15 → 26; both SKUs'get_analysis_factsdescriptions spell the list out (the Lite literal is the one Lite edit, a census-pin boundary extension)./* filled by lane N */.PgTargetScorer*.cs(0.0 /[]),PgTargetAdvice*.cs(null),PgTargetRelationshipGraph*.cs(empty chains),PgTargetFactCollector.*.cs(return immediately), detector calls, baseline metric map. The dispatchers in each family's root file are the entire contract; a lane's edit never leaves its own file.AllSqlis reflection-derived (public const string *Sql) rather than hand-listed likePgFactCollector.AllSql, because eight lanes write this collector's SQL in parallel and a hand list is the merge conflict every one of them hits. The convention it rests on is pinned: everynew NpgsqlCommand(in the family names such a const.PgTargetBaselineProvider : PgBaselineProviderwith oneprotected virtual ResolveBaselineQueryseam, inheriting the cache, the eight-column robust reader,RobustTierScaffoldand the ONEIsCommandTimeout(BaselineTimeoutIsNamedTests) — not a second provider copying that machinery to swap one switch.RelationshipGraphgains aprotected (bool buildSqlServerEdges)ctor andprotected AddEdge; the parameterless ctor is byte-identical (seven Lite tests construct it).PgTargetRelationshipGraphderives EMPTY — no SQL Server chain is reachable from a PostgreSQL fact (D2 made the keys disjoint), pinned.PG_SERVER_MAJOR_VERSION(Value = major, metadatais_aurora, base 0) — the twin ofSERVER_MAJOR_VERSION, read from the registry, emitted only when the major is known (never a fabricated 0). It is what lets a stub pass beemptyrather than tripping thefacts.Count == 0half of the empty-window rule, and it is the fact the D6 flavour disclosure will hang on.audit_configstops lying for a PostgreSQL target: it evaluated CTFP/MAXDOP/max-memory, found none, and said "no_config_data — the config collector may not have run yet" — false for a target whosepg_server_configcollector runs hourly. It now answersnot_collectednaming the tool's scope and pointing at theCONFIG_PG_*facts; a projection into its recommendations shape is the knobs lane's content.ToolRecommendationsroutes the three PostgreSQL prefixes to a pre-filledPgTargetToolRecommendationstable (whichget_pg_*read a family's table is a fact about the product, not a threshold), ahead of every SQL Server prefix arm.PgTargetThresholdLineageTestsscansPgTargetScorer*.csfor a numeric bar (ApplyThresholdFormula(, a space-delimited comparison,Boost =) without ameasured/engine-defined/unmeasuredmarker within six lines, and for any hour-in-milliseconds gate not keyed_per_hour. The scanner is exercised on arranged inputs so the rule is known to bite; it passes trivially on the stubs.Rejected: (a) a sibling
PgTargetAnalysisService— budget, abandon classification, persist, mute, notify would fork; (b) a sharedSource = "waits"— the D2 cross-fire; (c) catching inResolveEngineAsyncand defaulting to SQL Server — persists a "0 hours" verdict for a PostgreSQL target on a transient fault; (d) writing thePG_CPU_PERCENT >= 80load-family confirmer here — it is a bar, and bars are lane 9's under the lineage rule (the stub's comment names the arm).Every shared-file arm, so nothing else moved
FactScorer.cs:KnownSources+11;ScoreAll_ when PgTargetSources.IsPgSource(fact.Source) => PgTargetScorer.ScoreBase(fact);contextSources.UnionWith(PgTargetSources.All);IsDeviationScoredAnomalyKeyPgTargetScorer.IsDeviationScoredAnomalyKey(key) || …;ScoreAnomalyFactfirst lineif (PgTargetScorer.IsPgRatioAnomalyKey(fact.Key)) return PgTargetScorer.ScoreRatioAnomaly(fact);GetAmplifiers_ when PgTargetScorer.IsPgKey(fact.Key) => PgTargetScorer.Amplifiers(fact.Key)before theANOMALY_arm.InferenceEngine.cs:|| PgTargetFactKeys.IsConfigAdvisoryRoot(f.Key)in the entry-point filter.RelationshipGraph.cs:protected RelationshipGraph(bool),AddEdgeprivate → protected.FactAdvice.cs:Compose→PgTargetAdvice.Compose;GetForFactKey→PgTargetAdvice.Static, both ahead of the SQL Server prefix arms.AnomalyIncidentReconciler.cs:ResolveFamiliessecond lookup inPgTargetFactKeys.AnomalyToFamilies.PgFactCollector.Waits.cs:BuildCoverageprivate → internal.PgBaselineProvider.cs:ResolveBaselineQueryvirtual seam.PgAnomalyDetector/PgDrillDownCollector: implement the two new one-method seams (signature only).DarlingMcpTools.cs:audit_configPG arm;ToolRecommendationsPG prefix arm;FactSourceFilterDescription+11; rider (Brains-review campaign: deferred structural residue (from #3538 / #3539 / #3540 / #3541) #3653/The scorer's wait thresholds carry their measurement: WRITELOG stops saturating on a third of routine windows, deadlocks grade 5/hr apart from 90/hr, and the absolute gates scale with the hours actually observed (#3538 A5/A7) #3616): theHADR_SYNC_COMMITnext_toolsentry Lite got in Six small honesty riders from the brains-review residue: a retired knob says so on the wire, get_pg_server_config says when it was captured, Lite's latch/spinlock pages bind to the caller's limit, the ORDER-BY-integer fold gets a census, and two rung docs stop asserting superseded facts (#3653) #3659 —get_wait_trend {wait_type},get_ag_health(this SKU has the AG read; Lite points atget_alert_history),get_perfmon_trend Transactions/sec,get_file_io_stats.What it does NOT do
No schema (no rung, no
StorageVersionbump). No content: no PostgreSQL SQL beyond the coverage witness, the span gate, the baseline-data gate and the registry metadata read; no threshold; no advice text; no graph edge; no baseline CTE. No Lite twin of the collector (D1 — Lite has no PostgreSQL target). No change toPgFactCollector*orPgFactCollectorTests(32-name Lite-parity census untouched). The same-statement-pileup PG gate in the worker stays — that one IS SQL-Server-shaped (query_snapshots).FactRemediation.Build*stays inert for PostgreSQL keys (D8: no Apply/DDL).Blast radius
SQL Server pass: byte-identical, proven by running
analyze_server(2 findings),get_analysis_factsandcompare_analysisagainst the same planted store fromorigin/dev's binaries and this branch's, andcmp-ing the outputs (analysis_timenormalised): identical, 11,923 bytes each.AnalysisCoverageLivePostgresTests,AnalyzeWindowEmptyLivePostgresTests,DarlingAnalysisPipelineTests,PgFactCollectorTestsgreen on the live store. The only behaviour change reachable by a SQL Server target is the extra sentence on the insufficient-data message when — and only when — its registry row has NULLengine_kind.Test plan
New (compile-verified here; the ungated ones and the live ones were also EXECUTED on this machine through a net10.0 harness against a throwaway
timescale/timescaledb:2.28.1-pg18store migrated to the current ladder — 367 of the 368 Darling.Tests in the harness green, the one red being the harness's own csproj trippingDocCommentHygieneTests' project census; first CI execution is the Windows job):PgTargetFactCollectorTests— 12-name self-pin census + one-file-per-family +filled by lane N;AllSqlclosure (every const in the list, everyNpgsqlCommandnames a const); PG dialect pins; FROM/JOIN targets resolve to a collector table /servers/ CTE and never av_view; coverage SQL = SQL Server witness with one substitution, finished through the sharedBuildCoverage, first in emission order, no catch; deadline by reference + three-outcome degrade; metadata fact shape; gated e2e = the exit criterion (empty, coverage 1.0, 241 samples, onePG_SERVER_MAJOR_VERSIONfact;get_analysis_facts source=pg_config; a 2-hour sibling held by the gate on the right series).DarlingAnalysisServiceEngineRoutingTests—EngineForover everyMonitoredEngineKindtoken + NULL/unknown/case; two sets held for the service's life, distinct in every component; three entry points resolve before their first collector read, exactly three call sites, no cache field, no catch,CommandTimeoutset; both span SQLs; gated e2e for the NULL-kind row (SQL Server set, message names the stamp; stamped → PostgreSQL series).PgTargetSharedSwitchRoutingTests— vocabulary invariants;WaitKeynormalisation (9 cases); behavioural: a PG z-score anomaly scores 1.0 / 0.5 / 0 through the shared ramp, the PG ratio anomaly takes the PG ramp (0 today — routing, since the SQL Server arm would read 1.0), SQL Server anomalies unaffected; aCONFIG_PG_*root roots a card at 0.4 on the PG graph while the evidence-gated key does not; PG graph empty for every SQL Server root, parameterless graph unchanged, seam accessibility;FactAdvicedelegates every key andANOMALY_PG_WAIT_PROFILEnever reaches theANOMALY_WAIT_composer; reconciler foldsANOMALY_PG_DEADLOCK_RATEintoPG_DEADLOCK_RATEsame-db only.PgTargetThresholdLineageTests— the rule, the A7 rule, and the scanner's own fixtures (bare bar / comparison / boost flagged; each marker shape accepted; trivial literals, prose, literal text ignored; six lines in, seven out).PgTargetMcpSurfaceTests— every PG recommendation names a registered tool and onlyget_pg_*/get_analysis_*; every key has a row; story paths dedupe and never yield a SQL Server read;audit_configarm precedes the fact read; instructions namepg_database_stats; theHADR_SYNC_COMMITrider.Rewritten:
CollectorRunnerConnectionEngineTests.TheScheduledAnalysisPassIsGatedByEngine→…RoutesByRegistryEngine(no tombstone/latch/engine test in the tick window; threeResolveEngineAsynccall sites; no cache field).PostgresEngineGateBehaviorTests' twoanalyze_nowcases → the PG target RUNS the pass (empty fixture: generic gate message, never "does not apply"; planted 25 h:Ran,analysis_statecleared) and the SQL Server target's message is byte-identical; reflective latch helper deleted, pass-capable fabricated worker added.FactSourceRegistryTests15 → 26 +PgTargetSources.Allequality.FactCollectorCommandTimeoutTests/FactCollectorFailureReportingTestsglobs widened toPgTargetFactCollector.*.cs.Builds (zero warnings of ours):
PerformanceMonitor.Analysis,PerformanceMonitor.Darling.Analysis,PerformanceMonitor.Darling.Service,Darling.Tests,Lite.Tests,deprecated/Dashboard.Tests(-p:EnableWindowsTargeting=truefor the test projects). The one warning in the Lite.Tests build (LogTailOverlapThresholdPinTests.cs:95xUnit2000) is pre-existing on dev from #3646.Docs the change falsified, fixed:
Darling/README.md(two sentences),docs/postgres-first-target-runbook.md("what remains" bullet),DarlingMcpInstructions.cs(which series the 24 h is measured on, and that the pass routes byengine_kind).Which component(s) does this affect?
get_analysis_factssource description, a census pin)How was this tested?
Above. Live store: throwaway TimescaleDB 2.28.1 / PostgreSQL 18 container on Docker context
desktop-linux, migrated to the current ladder (V129 on dev after the rebase), torn down after. No production store or fleet server was read.Checklist
dotnet build -c Debug)Partial for #3542 (v1 plumbing; steps 2–9 follow). Rider for #3653/#3616 (
HADR_SYNC_COMMITnext_tools, Darling half of #3659's pair). CHANGELOG deliberately untouched — the coordinator splices this wave's entries.