Repository navigation
The PostgreSQL engine absorbs what waves 4–5 reported back: get_pg_database_stats surfaces numbackends, get_pg_cpu_utilization carries the host's memory, the spill leads to the memory composition check, ANOMALY_PG_CPU_BURN is the load family's stock second confirmer, autovacuum_enabled = off reads as off, the replication e2e is anchored, and four stale docs say what is true now (partial #3691, between waves) - #3832
Merged
Merged
Conversation
…tabase_stats surfaces numbackends, get_pg_cpu_utilization carries the host's memory, the spill leads to the memory composition check, ANOMALY_PG_CPU_BURN is the load family's stock second confirmer, the autovacuum collector reads autovacuum_enabled = off as off, the replication e2e is anchored on as_of, and four stale docs say what is true now (partial #3691, between waves)
erikdarlingdata
added a commit
that referenced
this pull request
Sep 21, 2026
…d fail: #3826's two informational store self-alerts were never named in the constant an operator reads, and the pin that would have said so ran on a cancelled check AlertReadFailureCounter.FleetScopedReads is the prose every collection-health surface concatenates — it tells an operator which reads can put a number in the instance total and in no server's count. #3826 added the plan dimension's TOAST slack read and the store checkpointer pressure read, both RecordReadFailure(null, ...) sites, and named neither. An operator reading a nonzero instance total against that list would have gone looking for a cause the list does not contain. The constant now names both. TheFleetScopedInventory_MatchesWhatTheCounterActuallyRecords derives its set from source rather than from a hand-maintained list, so it saw eleven against an asserted nine; the count is eleven, each new site is asserted present in the recorded set AND in the constant, and the Darling-only tally in the mute-rule comment is recounted from the source rather than carried forward - it said four when the answer was nine. This is not new breakage. dev has been red on `build` since #3832; #3826 introduced it and merged while its own `build` check was cancelled rather than red, so nothing ever reported it.
erikdarlingdata
added a commit
that referenced
this pull request
Sep 21, 2026
…tgreSQL anomaly-key census counted thirteen where fourteen keys are declared, and the fleet-scoped read inventory promised nine reads where eleven can fail (#3837) * The PostgreSQL anomaly-key census asserted thirteen while its own comment enumerated fourteen: lane 34 and lane 38 each bumped the same count to 13 for their own key, and git merged the two identical lines without a conflict (#3831 residue) The census reflects over PgTargetFactKeys' ANOMALY_PG_ literals, excludes AnomalyPrefix and counts what is left. Fourteen keys are declared; the comment above the assertion names all fourteen (five v1, three v2, blocking, the stock sampled wait profile, the two v3, lane 34's own-normal share, lane 38's database growth). The number said thirteen. Neither lane was wrong on its own: each added one key to a base of twelve and wrote 13. The lines were textually identical, so the three-way merge took them as one change rather than two, and the conflict that would have surfaced the arithmetic never appeared. * The fleet-scoped read inventory promised nine reads while eleven could fail: #3826's two informational store self-alerts were never named in the constant an operator reads, and the pin that would have said so ran on a cancelled check AlertReadFailureCounter.FleetScopedReads is the prose every collection-health surface concatenates — it tells an operator which reads can put a number in the instance total and in no server's count. #3826 added the plan dimension's TOAST slack read and the store checkpointer pressure read, both RecordReadFailure(null, ...) sites, and named neither. An operator reading a nonzero instance total against that list would have gone looking for a cause the list does not contain. The constant now names both. TheFleetScopedInventory_MatchesWhatTheCounterActuallyRecords derives its set from source rather than from a hand-maintained list, so it saw eleven against an asserted nine; the count is eleven, each new site is asserted present in the recorded set AND in the constant, and the Darling-only tally in the mute-rule comment is recounted from the source rather than carried forward - it said four when the answer was nine. This is not new breakage. dev has been red on `build` since #3832; #3826 introduced it and merged while its own `build` check was cancelled rather than red, so nothing ever reported it.
54 of 55 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Headline mechanism. The third between-waves shared-file batch of #3691 — what #3688 and #3737 were for the earlier waves. Waves 4–5 (lanes 25–36) each reported items they could not make inside their file-disjoint boundary, and lane 36's real-stock measurement (16/17 stock families reachable, four stories past 1.5, the load-anomaly pair capped at 1.30 exactly with
ANOMALY_PG_CPU_BURNfiring at 25σ beside them, unread) turned two of them into decisions. This PR decides and makes all nine as one change. PARTIAL for #3691 — do not close. The checklist lines it completes, verbatim: the twoPgTargetToolRecommendations/get_pg_database_statsresidue lines (numbackends surfaced on the database read; the memory rows now point at a read that carries memory) and the replication-flake line; item 7's line (the stock second confirmer) is enabled — the coordinator sent GO on lane 36's number.The lies it stops telling
get_pg_database_statscould not say how many backends were connected. V133 storednumbackends(pg_database_stats stores numbackends and pg_wait_sampling stores sampled_ms (Darling V133), so a PostgreSQL operator's saturation fraction has a numerator and the sampler arm's wait rate has its duty-cycle denominator (partial #3691) #3716) and the reader's own doc said "Not yet read here". The read now projects the window's peak per database (MAX(numbackends), carried through both CTEs — the parse census caught the first draft that did not), the row and payload carry it nullable (null= not sampled, every pre-V133 row; never 0 = idle), and the per-database peaks are deliberately not summed — they land at different instants, so their sum is a number nomax_connectionsfraction can honestly use; the description sends that question to thePG_CONNECTION_SATURATIONfact, which sums at one instant. Web column (Peak Backends), Viewer column (the projection census requires a column or a fold; a column, namedPeakNumbackendsso the census matches, renderingnot sampledfor a NULL — distinct from—, which means "the engine has no such figure"), README (§ tools, both migration-ledger rows whose "read by nothing yet" was already false — the saturation fact and the sampled wait profile read them since lanes 25/24) and runbook sentences.PG_HOST_MEMORY_PRESSURE/CONFIG_PG_MEMORY_OVERCOMMITtool rows nameget_pg_cpu_utilization(the V136 memory columns ride the capacity row), and PostgreSQL targets get a per-database size series (pg_database_size_stats, hourly, a year) and the host's memory as six byte columns on pg_cpu_utilization's Performance Insights row (Darling V136), so the object-growth and memory-composition lanes have a source (partial #3691) #3794's census pinned the collector as the ONLY reader — so the tool answered with CPU alone.DarlingPgCpuUtilizationReader.HistorySql(the served read) now selects the six columns into aHostMemoryrecord onCpuSample; the two ALERT reads (LatestCpuSql,SamplesSinceSql) select exactly what they did before V136 — the High CPU gate is a sub-second alert-path read with no memory question.CpuSample.Memoryis not defaulted (the capacity trio's rule: a null is stated at every construction site — the gate's two sites sayMemory: null— never omitted; a row read WITH memory columns all NULL is a non-null record with null members, so the two states stay distinguishable). The tool reports the six per bucket in whole bytes (MeanBytes, null stays null — zero bytes of host memory is a measurement nobody made) plusmemory_samples_in_bucket, its own denominator on the capacity pattern; the description names every key. PostgreSQL targets get a per-database size series (pg_database_size_stats, hourly, a year) and the host's memory as six byte columns on pg_cpu_utilization's Performance Insights row (Darling V136), so the object-growth and memory-composition lanes have a source (partial #3691) #3794's positive census becomes a two-member roster (collector +HistorySql, each asserted to name all six; the tool file named as the payload projection; the two gate reads asserted to name none).PG_TEMP_SPILLas one of the sum's two D5 co-fires and wrote the amplifier, but the spill's edges live inPgTargetRelationshipGraph.Query.cs(one source node's edges in one place), so the story never opened (PostgreSQL targets learn whether the memory configuration fits the host: the §4b worst-case sum against the window's smallest memory_total_bytes roots an advisory at 0.4 and reaches the incident line only when host-memory pressure or temp spills co-fire, the host's sustained reclaimable share is graded, and a stock target reads unavailable (partial #3691, lane 32) #3809). Declared there now beside thework_memedge, under the temp chain's category, base-gated on the sum's verdict (BaseSeverity > 0, the scorer's arithmetic with its lineage — never a ratio of its own; an inflatedSeverityon a base-0 sum opens nothing). Inert on stock by construction (no host-memory source → no sum fact). Traversal picks the higher destination and skips a consumed leaf, so the live memory e2e's outcome is unchanged and now pinned: pressure outranks the spill, claims the sum, and the sum sits in exactly one path.PgTargetTempTests'Singlebecame "the work_mem edge is one of them";PgTargetMemoryTests'DoesNotContainflipped to "exists, and is not this file's", each saying why.ANOMALY_PG_CPU_SPIKE, the measuredPG_CPU_PERCENT) are Aurora-only, so an extreme TPS or session anomaly could never find a second corroborator on stock — whileANOMALY_PG_CPU_BURN(cores busy frompg_stat_kcacheagainst the server's own hour-of-week routine, the same surge statement from the kernel's counters) fired beside it. The burn anomaly joins the load family's confirmers (LoadAnomalyAmplifiers), takes the load arms in return (symmetry, never itself), andAnomalyToFamilies[AnomalyCpuBurn]addsCpuDecompositionbeside the base-0CpuBurnCoresso the fold has a positive-base parent to land on. Pinned: TPS extreme + session + burn → 1.6 (pages); without the burn → 1.30 (lane 36's number); a burn that did not fire confirms nothing (the predicate reads the base); the burn as root with two siblings → 1.6 with four arms and never its own. Arm counts moved 3→4 (load), 5→6 (both wait profiles) with the reason.autovacuum_enabled = offread as ENABLED.PgAutovacuumStatsCollector.cs:143,161testedlower(option_value) = 'false', butpg_options_to_tablereturns the reloption literal as typed —off(the docs' own spelling),0,no,f,n, and any prefix PostgreSQL'sparse_boolaccepts (fal). Lane 36 measuredautovacuum_disabled = fon all 47 rows of a table switched off withoff, soCONFIG_PG_AUTOVACUUM_DISABLED(A PostgreSQL table that turned autovacuum off and fell behind is its own card: CONFIG_PG_AUTOVACUUM_DISABLED names the table, how far past its own trigger line it sits and for how long, and joins the backlog's story on the same table (#3691 step 22) #3761) could never fire for it. Fixed in both places the flag is computed (the projected column AND the activity filter's keep clause — fixing one would have kept the table out of the rows while the column said disabled) asNOT option_value::boolean: the exact predicate rather than a hand list, because the server validated the literal with the boolean input function when the option was SET, so every value the catalog can hold casts and every spelling of off reads as off (falis the case an IN list misses — verified live on PG 18, all seven spellings, before and after). Collector-file boundary extension —PerformanceMonitor.Collectors/PgAutovacuumStatsCollector.cswas outside the shared-file set; the coordinator granted it as item 8. Unit pin inLite.Tests(2 cast sites, no= 'false'left), and a new live class that plants temp tables with all six off spellings +on+ no reloption inside one rolled-back transaction and runs the collector's real statement: every off spelling admitted AND flagged,onand plain not admitted.windowEndwas the fixture's clock;analyze_serverwithoutas_ofused its own wall clock, so its window started 1 min + elapsed seconds afterwindowStart; crossing a minute boundary between the two dropped the first sample (windowStart + 2 min), the lag fact read 47 samples, and "48 samples of this standby" went red then green on re-run. Both tool calls now passas_of: windowEnd— the tool's window IS the collector context's. Run 6× locally including one started at :58 across the boundary — all green.SeqScanAdvisory("never CREATE INDEX (D8), the standing no-missing-index-folklore rule" — a misattribution, see Restore missing_indexes[].create_statement on the plan tools — #3696 dropped it citing a 'no-missing-index-recs rule' that was never made; DDL recommendations are legitimate product output (RCSI precedent) #3805) → the maintainer's 2026-09-20 index-evidence discipline (evidence first; candidate with its cost only wherepg_qualstatsnamed the columns; impact labelled one operator's estimate; regression caveat on every card; advisory band, never a defining characteristic).PgBaselineProvider.ResolveBaselineQuery"the one seam" → the first of three (clock, keyed;PgTargetClockTestscounts them). Both Plans docs "(a) the seam keys one series per (server, metric)" → the keyed seam exists since A baseline series can now be scoped to ONE statement: the per-key seam on PgBaselineProvider (a keyed GetBaselineAsync overload, a third resolve seam, $7 after the Q6 clock) and two per-queryid arms — a statement's share of the collection's execution time and its per-call mean — so a PostgreSQL bad actor can be judged against its OWN hour-of-week habit; no consumer yet (partial #3691, lane 33) #3810, switching to per-queryid is lane 27's follow-up.PgTargetAnomalyDetectorclass summary gains lane 35's sentence:HasBaselineDataSqlgates the WHOLE pass onpg_database_statshaving 30 d of rows — a target whose database-stats collector fails gets no anomaly of ANY family, by design.PgTargetV3PlumbingTestsclaimed inertness over an empty set. Renamed to what it still pins (…AreFilled_SelfGateOnBareFacts_AndRouteTheMemoryCheckByNameAheadOfThePrefixArms) with the summary rewritten; the old name is asserted absent.Censuses that fired, and how each was satisfied
DarlingPgReadSqlParsesLiveTests(caught the CTE omission — fixed, not exempted);PgNumbackendsAndSampledMsRungTests.NoReaderSelectsEitherColumnYet…→ re-shaped as the consumers-landed pin (MAX, no LAG, no SUM; the reader doc's new sentences);PgDatabaseSizeStatsAndHostMemoryRungTests(two-member roster, above);ViewerPostgresTabsTests.EveryFieldTheSharedReadersReturn…(a column, not a fold);PgTargetTempTests/PgTargetMemoryTests/PgTargetKernelTests/PgTargetV3PlumbingTests/PgTargetSharedSwitchRoutingTests/PgTargetAnomalyTests/PgTargetSampledWaitTestspins moved with the fold and the arm counts, each saying why. Green with no edit:DocCommentHygiene,RepoFileAdoption(single-line anchors,ReadRepoFile),AuroraOnlySqlIsGated,StoreSqlClockDiscipline,LiveCleanupConversionRatchet+LivePostgresCollectionHygiene(the new live class rolls back — nofinally, no store write survives),BaselineSupply,LocalClockBucketKey,FleetIdentifierScrub,CommentFilterAdoption,McpPageContract,FactSourceRegistry,MeasurementContractCensus,AsOfWindowAnchor,McpReadCommandTimeout,ServerPageTabs,ViewerCollectorCoverage.Blast radius
SQL Server pass: byte-identical — no shared switch changed; the routing pins pass unchanged. PostgreSQL pass: an extreme load anomaly beside a fired burn anomaly pages where it read 1.30; a spilling Aurora host whose pressure is quiet may now tell spill → sum (previously spill → work_mem or alone); the burn anomaly folds onto the decomposition incident;
autovacuum_disabledis TRUE for the docs' spelling from the next collection (the fact and the advice were already written for it). Alert path untouched (LatestCpuSql,SamplesSinceSqlbyte-identical; the gate constructsMemory: null). No schema, no rung, noStorageVersionbump (V137 is dev's).What it does NOT do
No Lite twin (no
pg_*table in DuckDB). No backfill ofautovacuum_disabledon stored rows (the next collection rewrites it). No per-queryid plan anomaly (lane 27's follow-up, now correctly described). No Viewer rendering of the memory columns (the memory family's tool rows point at the MCP read; the Viewer's CPU tab is unchanged). Does not arm auto-merge (review-dark window); landing order is the coordinator's.Test plan
New:
Darling/Darling.Tests/PgTargetBetweenWavesV3Tests.cs(8 tests: memory shape + description + payload keys; the edge, its gate, its file and the two story shapes; the four docs; the e2e'sas_of; the rename; the 1.6 / 1.3 / quiet-burn / symmetry pins; item 9),PgAutovacuumDisabledSpellingLiveTests.cs(live, rolled back),DarlingPgDatabaseReaderTests.NumbackendsIsProjectedAsAPeakLevel…,Lite.Tests…DetectsAutovacuumDisabledPerTable_ThroughTheBooleanCast_NotAStringCompare. Executed here, not only compiled — a net10.0 harness (AssemblyName=Darling.Tests, the whole Darling.Tests tree minus the WPF-only classes, Viewer dll referenced) against a throwawaytimescale/timescaledb:2.28.1-pg18on Docker contextdesktop-linux, migrated by the fixture: 1148 / 1148 green across everyPgTarget*Testsclass, the censuses above and every touched reader/tool test, live e2es included (memory Aurora + stock, replication ×6, vacuum, the new spelling class); the Lite pin executed in a Collectors-only net10.0 harness (17 / 17). Builds, zero warnings:PerformanceMonitor.Analysis,PerformanceMonitor.Collectors,PerformanceMonitor.Darling.Service,PerformanceMonitor.Darling.Viewer(EnableWindowsTargeting),Darling.Tests,Lite.Tests. CI's Windows job is the first execution of the Viewer XAML column and the Lite test as shipped.Which component(s) does this affect?
How was this tested?
Above. No production store or fleet server was read; the rig was a local container, torn down.
Checklist
dotnet build -c Debug)Partial for #3691 (between waves, batch 3). CHANGELOG deliberately untouched — the coordinator splices this wave's entries.