Repository navigation
PostgreSQL sessions family: an operator's connection-saturation card now divides pg_stat_database.numbackends — the universal one-minute level, summed over databases at one instant, peaked over the window — by the ceiling, and names the exception-capture peak as its stated fallback; both readings ride the fact, the bars do not move (partial #3691, lane 25) - #3782
Merged
Merged
Conversation
…ceiling — the universal one-minute level, summed over databases at one instant, peaked over the window — and names the exception-capture peak as its stated fallback; both readings ride the fact, the bars do not move (partial #3691, lane 25)
…ends/ceiling read on 2026-09-20 (p99 median 1.8 %, fleet max 10.0 % over 20 h × 50 clusters) agrees with the capture peak within 10 %; the bars stand (partial #3691, lane 25)
This was referenced Sep 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Headline mechanism. A PostgreSQL operator's connection-saturation card now divides the right number by the ceiling.
PG_CONNECTION_SATURATION's numerator moves frompg_session_states.total_sessions— the exception-capture peak — topg_stat_database.numbackends(V133, #3716): the collector's one-minute LEVEL of backends attached to a database, summed over the databases at onecollection_time, then the window's peak of those instants. Where the store does not carry the column yet — a pre-V133 store, or one mid-migration — the capture peak is used exactly as v1 did, and the fact says which (numerator_source). Both readings ride the metadata whenever both exist;Valueis whichever decided; no bar moves. Lane 25 of #3691.PARTIAL for #3691 — do not close. Completes the consumer half of this checklist line, quoted verbatim:
numbackendsonpg_database_stats— "(#3716, V133 — stored; the saturation numerator switch is a consumer lane once the column has data)". The column has ~20 h of fleet rows as of 2026-09-20 13:00Z.The lie it stops telling
v1 had one numerator and it was an exception table's.
pg_session_statesstores a capture only when some session tripped a rule, so its peaktotal_sessionsis a peak over the minutes that had something to report: biased HIGH on a server that captures often, blind on one that never does — the fleet's captures per server ran 59 – 2,100 over 7 days, wildly uneven — and it counts PostgreSQL's own database-less background processes, which hold nomax_connectionsslot. Design §3.6 wantednumbackendsfor this from the start; the rung landed yesterday.numbackendsis universal (every server, every minute), a level (never differenced,stats_reset-agnostic), and attached-to-a-database backends only.What changes, and from which table
Darling/PerformanceMonitor.Darling.Analysis/PgTargetFactCollector.Sessions.cs— one new read,PgTargetNumbackendsPeakSqloverpg_database_stats(a collector table already in the FROM/JOIN whitelist), three CTEs:instants=SUM(numbackends) … GROUP BY collection_time;shape= how many instants the window holds and how many carried the column (COUNT(*) FILTER (WHERE numbackends IS NOT NULL)— coverage in INSTANTS, not rows);peak= the largest sampled instant.LEFT JOIN … ON trueso the shape row survives a window with no sampled instant. NoLAG, nostats_reset— a level is never differenced. It runs on the same connection, under the same deadline and degrade, after the ceiling lookup, so a window this family would not grade costs no third query (pinned by code order).The numerator decision is a pure public function,
ChooseSaturationNumerator(databaseStatsSamples, numbackendsSamples)→(UseNumbackends, Partial): no sampled instant → capture peak, not partial (a pre-V133 store — nothing the column could have covered); coverage ≥NumbackendsCoverageFloor(½) → the level; SOME sampled but under half → capture peak withnumbackends_partial = 1(a store mid-migration, or a collector restarted onto the rung inside the window — a peak over the sampled half of a window is the capture-peak lie in a new coat).Metadata, stamped on every saturation fact:
numerator_source(1 = numbackends, 0 = capture peak),numbackends_partial(0/1),numbackends_samples,database_stats_samples; and whenever at least one instant carried the level — INCLUDING the partial fallback —peak_numbackendsandnumbackends_peak_age_s.peak_total_sessionsstays under its v1 name whichever numerator decided: #3713's compare banding reads it, and the state breakdown (active / idle in transaction / other) is the capture's and only the capture's.peak_age_sstays the capture peak's age; the level's instant has its own key.Two instruments, two denominators, stated. The idle-in-transaction SHARE (
peak_idle_in_transaction_share— lane 3's amplifier gate and the graph edge's) keeps the capture's owntotal_sessionsas its denominator.numbackendshas no state; a share of it would be a count from one instrument over a count from another. The collector doc, the scorer doc and the advice all say so.PerformanceMonitor.Analysis/PgTargetScorer.Sessions.cs— constants and prose only. The eight metadata-key constants,NumbackendsCoverageFloor = 0.5(definitional — theRedactedShareMajorityshape: a column arrives with a rung and from then on every row has it, so a real store reads ~0 or ~1 and there is nothing to calibrate), and the doc updates below.ScoreSessionsFactis untouched: it gradessaturation_ratiobetween the same bands from either numerator.PerformanceMonitor.Analysis/PgTargetAdvice.Sessions.cs— the numerator names its instrument. Whennumbackendsdecided: "At the window's peak minute, 1.5 h before the window's end, 93 client backends were connected against 97 usable connections — pg_stat_database.numbackends summed over every database at that one instant, against max_connections 100 minus superuser_reserved_connections 3 — that is 93 / (100 − 3) = 96%.", headline "Connections peaked at 96% of the usable ceiling (93 client backends of 97) — …"; the capture is introduced as what it is — "State breakdown at the peak session capture (40 sessions, 2 h before the window's end): 15 active, 5 idle in transaction, 20 other …" — with one sentence saying the numerator is the level, not that capture, and why; the coverage sentence names how many of the window's samples carried it; the capture-only caveat ("reads a few points high") is NOT claimed of the level; the PARKED sentence and the lever sentence say "of the peak capture". When the capture peak decided: lane 3's block byte for byte plus ONE sentence — "The numerator is the peak of the session captures (exception-driven; numbackends not yet collected on this store)." — or, mid-migration, "… pg_stat_database.numbackends is present on only 101 of the window's 241 pg_database_stats samples — a store mid-migration — so the level is not used until at least half of them carry it; over the samples that do, it peaked at 93." The static block names both instruments and the rule between them and still claims no figure.The lineage shape of every bar
ConnectionSaturationWarning/CriticalSUM(numbackends)/ceiling per-server p50 median 0.8 %, p99 median 1.8 %, fleet max 10.0 % (one cluster); the capture peak over the SAME window p99 median 2.0 %, fleet max 10.0 % — the two numerators agree within ~10 % where both exist, from 1,056numbackendssamples per server against a median 24 captures. The bars 0.8 / 0.9 stay engine-defined and deep in the measured empty interval;threshold_lineagestays 1. Today's 24 h coverage is 86 % on every server (the pre-V133 hours), so the ½ floor passes now and reads 100 % from tomorrow.NumbackendsCoverageFloorIdleInTransactionShareBartotal_sessions, on purposeThe C1 read landed after open (second commit): the scorer's
ConnectionSaturationWarningblock, the class summary and the advice's trailing clause carry the 2026-09-20 numbers; the fallback arm's text is unchanged (byte-identity pin still holds).Co-fire gating
Unchanged. The three saturation amplifiers (parked share, CPU fired, offered-vs-delivered) read the same keys they did; the parked-share arm's gate is the capture's share. The graph edges (
PgTargetRelationshipGraph.Saturation.cs) are untouched. A saturation fact at the same ratio scores and amplifies the same whichever numerator produced the ratio (pinned).What it does NOT do
numbackendscould now give it a numerator — the state breakdown, the redaction verdict and the newest-capture figure all come from the captures, and emitting without them changes what the card says. That is the honest NEXT step (a level-only card shape), stated in the collector doc, not this one.PG_MONITORING_PERMISSIONSstill replaces the ratio when the login could not see state, althoughpg_stat_databaseis unprivileged andnumbackendsis readable there. Same reason: the family's other claims need the captures. Reported below as the coordinator's call.PgTargetFactKeys, touch the graph, the scorer's grading,FactScorer,FactAdvice, any root partial,PgSessionStatesCollector,PgDatabaseStatsCollector, or a rung (StorageVersionat V135 on dev). No production read.CollectSessionFactsAsync, behind thepg_sessionssource).Two claims from the rung's doc, checked on a live engine
Measured on the throwaway PG18 store (
SELECT datname, numbackends FROM pg_stat_database;pg_stat_activitygrouped bybackend_type):numbackendsis0, not the NULL pg_database_stats stores numbackends and pg_wait_sampling stores sampled_ms (Darling V133), so a PostgreSQL operator's saturation fraction has a numerator and the sampler arm's wait rate has its duty-cycle denominator (partial #3691) #3716's doc expected ("NULL where the engine reports none (the shared-relations row)").pg_stat_get_db_numbackends(0)reports 0 on PG18. Harmless to this read — 0 sums as nothing, and NULL would too — and the collector's own version audit is right that the column exists; only the row-contract sentence is off. The collector doc here says what was measured; the live fixture plants the shared row as 0. Reported for the coordinator.darlingread 2 backends with ONE client connected: the TimescaleDB per-database scheduler worker counts.numbackendscounts backends ATTACHED TO A DATABASE — client backends plus autovacuum / parallel / logical-replication / extension workers while inside one — and none of the database-less processes (checkpointer, walwriter, bgwriter, launchers, physical WAL senders, I/O workers) the capture'stotal_sessionscounted. Workers hold nomax_connectionsslot either, so the ratio can still read a hair high during a vacuum or a parallel plan — the safe direction, and far less of it than before. The collector doc and the advice state it that way rather than as abackend_typefilter the view does not offer.Test plan
Darling/Darling.Tests/PgTargetSessionsTests.cs(+4 unit facts / theories, 1 existing pin re-counted 2 → 3 commands and renamed, the existing e2e extended with a fourth server):ChooseSaturationNumerator_…(6 rows): (0,0) and (241,0) → capture, not partial; (241,241) → level; (240,120) → level (half IS the floor); (240,119) and (241,101) → capture + partial; the floor is the scorer's one constant.TheLevelStamps_PutBothReadingsOnTheFact_AndValueFollowsWhicheverDecided_WhileTheBarsDoNotMove: all sampled →Value93/97,peak_numbackends93 ANDpeak_total_sessions40, idle share 5/40 (the capture's),ScoreBase1.0 / lineage 1 / severity 1.0 with no arm matched through the realScoreAll; partial → 40/97, source 0, partial 1, the unused 93 still shown, scores 0; none → 90/97, no level keys; the six key strings by name.ComposeSessions_NamesTheLevelAsTheNumerator_…: the headline and the first sentence by exact string; the capture introduced at its own instant and count with the unchanged split; "The numerator above is the level, not that capture"; the coverage sentence; the pinned measured clause AND its second sentence; the capture-only caveats absent; over the parked bar the share and lever say "peak capture".ComposeSessions_StatesTheCapturePeakAsTheNumeratorWithItsReason_…_AndAPreV133FactIsByteIdentical: lane 3's sentences byte for byte plus the one fallback sentence;Assert.Equal(preLane, block)— a fact with NO level keys composes the same block as one stamped "capture peak, nothing sampled"; the mid-migration sentence by exact string; the static block's two instruments.TheSessionReads_AreThreeQueriesOverCollectorTables_…(wasTwoQueries): the new const inAllSql;FROM pg_database_statsonly;SUM(numbackends)BEFOREGROUP BY collection_timeand exactly oneSUM(; the peakORDER BY numbackends DESC, collection_time DESC; noLAG(/stats_reset; theIS NOT NULLcoverage and peak filters;LEFT JOIN peak AS p ON true; the level read AFTER the ceiling lookup in code order;ChooseSaturationNumerator(level.…)and the three key constants named in code;["peak_total_sessions"] = peakTotalretained; 3 commands / 3 deadlines.Gated e2e (the existing
APlantedSeries…test, fourth server…-level): 25 h of span plus every minute of the 4 h window as the V133 collector writes it —appdb20 +reports10 + the shared row's 0 = 30, and ONE instant 90 min before the end at 60 + 33 = 93 — beside 48 captures peaking at 40 (15 / 5 / 20). The collector alone:Value93/97, source 1, partial 0,peak_numbackends93 at 5,400 s, 241 / 241 instants,peak_total_sessions40 at 7,200 s, the split, idle share 5/40; scored 1.0 / 1.0 / lineage 1. Through the REALanalyze_serveranchored at the window's end: the card at 1.0 with the exact headline, the first sentence, the capture-breakdown sentence, the coverage sentence, no fallback sentence;get_analysis_factsshows source 1, 93 and 40 side by side. Then the same store mid-migration (UPDATE … SET numbackends = NULLon every instant older than the last 100 min → 101 of 241): 40/97, source 0, partial 1,peak_numbackendsstill 93, the mid-migration sentence. Then NULL everywhere: 40/97, source 0, no level key, the not-collected sentence — v1's reading. The busy server (the shared planter's NULL column) additionally asserts source 0 / 0 of 241 / no level key, and lane 18's 31-day e2e (NULL column) passes unchanged — the byte-identity claim, executed.Executed here, not only compiled: a net10.0 harness (
AssemblyName=Darling.Tests, 20 PgTarget census / hygiene classes incl.PgTargetFactCollectorTests,PgTargetSharedSwitchRoutingTests,PgTargetThresholdLineageTests,PgTargetMeasuredLineageTests,StoreSqlClockDisciplineTests,DocCommentHygieneTests,RepoFileAdoptionTests,FleetIdentifierScrubTests,FactCollectorCommandTimeoutTests,FactSourceRegistryTests,LiveCleanupConversionRatchetTests,LivePostgresCollectionHygieneTests,PgTargetBetweenWaves*,PgTargetWaitTests,PgTargetNamedHopsTests,CollectionCaveatsTests) under<worktree>/obj/mactest/against a throwawaytimescale/timescaledb:2.28.1-pg18on Docker contextdesktop-linux(role and databasedarling, migrated by the fixture to V135), torn down after: 283 / 283 green withDARLING_TEST_PGset — every gated e2e in the set ran (183kpg_database_statsinserts on the store), including this lane's and lanes 3 / 14 / 18's. First execution of every other census is CI's Windows job.Builds (
-c Release -p:EnableWindowsTargeting=true):PerformanceMonitor.Analysis,PerformanceMonitor.Darling.Analysis,PerformanceMonitor.Darling.Service,deprecated/Dashboard.Tests— 0 warnings;Darling.Tests— one pre-existing xUnit2013 (ConfigChangeTraceAnchorTests.cs:94);Lite.Tests— the two pre-existing (LogTailOverlapThresholdPinTests.cs:95xUnit2000,DefaultTraceEventsCollectorDefinitionTests.cs:211xUnit2013). None in a file this PR touches.Which component(s) does this affect?
How was this tested?
Above. Throwaway container only; no production store or fleet server was read.
Checklist
dotnet build -c Debug)pg_sessionsfamily)Partial for #3691 (lane 25). CHANGELOG deliberately untouched — the coordinator splices this wave's entries. Not armed (review-dark window).