Skip to content

pg_database_stats stores numbackends and pg_wait_sampling stores sampled_ms (Darling V133), so a PostgreSQL operator's saturation fraction has a numerator and the sampler arm's wait rate has its duty-cycle denominator (partial #3691) - #3716

Merged
erikdarlingdata merged 1 commit into
devfrom
feat/3691-v133-pg-columns
Sep 19, 2026

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

The saturation fraction had no numerator and the sampler arm's wait rate had no denominator

Partial for #3691 — the ONE migration rung for this wave; V134 is free after merge. Darling rung V133 (pg-numbackends-and-sampled-ms). Numbered max(dev) + 1 at open and re-checked against origin/dev immediately before push (V132 was still the top; nothing else is in flight). Completes these #3691 checklist lines VERBATIM:

  • numbackends on pg_database_stats — a RUNG (universal 1-min client-only numerator for saturation instead of exception-capture peaks). Ladder-sequenced with the other seat.
  • the sampled_ms half of Stock PostgreSQL's sampled waits get no anomaly baseline … "or store sampled_ms per row (rung)" — the rung half only; the collector_state whitelist alternative is not taken and the consumer's arithmetic is not changed.

Does not close the issue.

The two lies the store told

Connections. The v1 saturation read had pg_session_states.total_sessions and nothing else to put over max_connections. That collector stores the sessions holding an open transaction old enough to matter — the exceptions, not the population — so a quiet server has no row, a busy one has whatever crossed the capture rule, and "count / max_connections" is not a saturation fraction. pg_stat_database.numbackends is the number the question wanted all along: client backends connected per database at the instant of the read, a LEVEL sampled every minute with the rest of the row, present since PostgreSQL 8.x (the oldest column in the view; the collector's version audit records it and the AppliesTo stays unconditional). It was never selected.

Waits. The #3604 service-side sampler stores profile_period_ms = 1000 and a cumulative tally, exactly like the extension arm — but it observes SamplerSnapshotsPerCycle one-second snapshots (30 s) of every 300 s cycle. A rate read over the interval naturally writes Δsamples × period / interval, which on that arm is a ~10× understatement, and nothing in the row said so: the instrument token that discloses the arm lives in collector_state, outside the analysis FROM/JOIN whitelist, and the period alone cannot say how much of the interval was watched.

The row contract

Table Column Sampler / collector writes Extension arm writes Pre-V133 rows Readers today
pg_database_stats numbackends integer NULL the level (d.numbackends, selected LAST) or NULL where the engine reports none (the shared-relations row) — NULL none
pg_wait_sampling sampled_ms integer NULL snapshots actually read × SamplerPeriodMs — 30,000 on a full window, counted by result-set shape like the tally itself NULL — the module samples the whole interval in-engine; there is no duty cycle to disclose NULL none

NULL means "not sampled", never 0. Zero connections is a measurement; a pre-rung row never took one. NULL sampled_ms means "the whole interval was observed", which is exactly today's samples × period arithmetic — so the extension arm and every historical row read as they do now the moment a consumer divides by it.

sampled_ms is per row, not a cumulative in state — the one design choice worth arguing. A cumulative observed-ms counter would mirror sample_count's reset-aware differencing, but it needs a third state key kept in step with the tally, and the tally's cap-drop "start over" semantics would decouple one key's samples from the shared denominator. The observed window of one collection is a fact only that collection knows and nothing can recover afterwards; a per-row value survives a tally reset or a cap-drop with no second counter; every row of a collection carries the same figure (like profile_period_ms), so a consumer wanting a window's total sums one value per collection_time between its endpoints. Stated in the rung doc, the collector's column comment and DarlingPgWaitSamplingReader.EstimatedWaitMs's doc, which now names sampled_ms as the denominator a rate read must use and says plainly that this reader does NOT yet make the correction.

What changed, and why this shape

Schema. PgMigrations V133: ALTER TABLE collect.pg_database_stats ADD COLUMN IF NOT EXISTS numbackends integer; + the same for collect.pg_wait_sampling.sampled_ms. Nullable, no DEFAULT, no backfill (V127/V128/V132's shape; both tables are compressed hypertables on the fleet and a DEFAULT is the compressed-segment rewrite trap). No view to refresh — the PostgreSQL collector tables have no v_ passthrough, so unlike V132 this rung is two ALTERs and nothing else (pinned: no VIEW in the SQL, neither table in AllPassthroughViews). The V101 rule: V83Sql and V96Sql CREATE texts carry the new columns too, because PgSchemaGeneratorTests.EveryPostgresRung_IsIdenticalToTheGeneratedSchema holds every collector rung column-for-column equal to the generator's CURRENT output — the fresh population builds the table from the generator at V1, the upgraded population gets it from this rung's ALTER; neither text is redundant, and both rung docs say so. StorageVersion 133.

Lite is unchanged, and that is a checked claim. Lite stores no pg_* table (DuckDbSchemaGenerator.StoredCollectors is the SQL Server set); V128's Lite bump was for the SQL Server families it also dressed, not for pg_wait_stats. Pinned by LiteStoresNeitherTable_SoTheRungHasNoDuckDbTwin (both collectors' TargetEngine is PostgreSQL).

Collectors. PgDatabaseStatsCollector: selects d.numbackends LAST (after stats_reset, so the reader ordinal 9 and the appended column agree), Row gains int? Numbackends, PayloadColumns appends numbackends Integer, WritePayload writes it tenth; the type header gains the "one level in a row of counters" paragraph and the version audit records the column. PgWaitSamplingCollector: Row gains int? SampledMs, PayloadColumns appends sampled_ms Integer, ReadSamplerAsync counts four-column result sets and stamps snapshots × SamplerPeriodMs on every emitted row, the extension arm's ReadAsync stamps null, WritePayload writes it seventh; the FLOOR paragraph in the type header says why.

Viewer probe (ViewerDataService.cs, the boundary extension every rung makes): the V133 COLUMN sentinel (pg_database_stats / numbackends — the rung's first ALTER, stated so nobody looks for a significance), ordinal 108, hasPgNumbackendsAndSampledMs top arm returning 133, V132's arm re-worded as one rung behind. The arm's prose states that the gate rests on the standing invariant alone (no viewer read names either column yet). Without it RequiredStoreSchemaVersion == StorageVersion.SchemaVersion would map every current store one rung low and the connect-time gate would refuse it.

Readers: doc only. DarlingPgWaitSamplingReader.EstimatedWaitMs — the addendum above, arithmetic untouched. DarlingPgDatabaseReader — a class-doc paragraph naming numbackends as the level it does not yet select and what a consumer must do (never difference it; NULL is "not sampled"; sum across databases for the cluster figure).

Docs. Darling/README.md: a V133 row in the migration table; the pg_database_stats and pg_wait_sampling rows in the collector table gain one sentence each ending "read by nothing yet".

Blast radius

  • Store: one nullable integer column on each of two PostgreSQL collector tables; catalog-only on PostgreSQL; verified on TimescaleDB 2.28.1 / PG18 below. SQL Server-only stores carry two more empty columns on two empty tables.
  • Write path: every pg_database_stats row gains the level (or NULL on the shared-relations row); every sampler-arm pg_wait_sampling row gains 30,000 (or the snapshots actually read × 1,000); every extension-arm row gains NULL. No existing column changes meaning or position.
  • Reads, MCP, viewers, analysis, alerting, CAGGs: untouched and unaffected — no product read names either column (pinned: NoReaderSelectsEitherColumnYet_… greps both readers' SQL constants). The PgTarget* analysis files are not touched; the existing differencing read over rows carrying the new columns is exercised live below and reports what it always did.
  • Row types: two collector Row records gain a trailing nullable field; the two Lite fixtures that constructed PgDatabaseStatsCollector.Row positionally are updated.

What this PR does NOT do

  • Does not close PostgreSQL-target analysis engine: v2/v3 slices and the v1 lanes' residue (from #3542) #3691. Does not read either column: no saturation fraction, no duty-cycle-corrected wait rate, no numbackends on get_pg_database_stats, no Compose measure. Those are the consumer lanes once the fleet has a day of rows.
  • Does not whitelist collector_state for analysis reads (the checklist line's other alternative) — the row now carries what the token would have said.
  • Does not backfill, touch pg_session_states, change EstimatedWaitMs's arithmetic, or touch Lite, deprecated/Dashboard, install/ or any PgTarget* file.

Live verification

PostgreSQL 18 + TimescaleDB 2.28.1 (timescale/timescaledb:2.28.1-pg18, throwaway container on Docker context desktop-linux, torn down):

  • PgNumbackendsAndSampledMsLivePostgresTests (gated, new): MigrateAsync to 133; then the store is put one rung short (both columns dropped, the 133 stamp deleted) and MigrateAsync applies exactly one rung and then zero, stamping 133; both columns read integer / YES / no default and are the LAST column of their tables; one populated row and one NULL row per table through each collector's real WritePayload over a real NpgsqlBinaryImporter (DarlingCollectorRunner's COPY loop in shape) read back verbatim (("appdb", 37), (NULL, NULL); Lock rows 30,000 and IO rows NULL across two cycles); DarlingPgWaitSamplingReader.GetPgWaitSamplingPageAsync over those rows differences as before (Lock Δ7 → EstimatedWaitMs 7,000; IO Δ1,000 → 10,000) and publishes neither column.
  • PgWaitSamplerLiveTests (gated, changed): the REAL sampler batch through the real runner against a live target — every stored row now also asserts sampled_ms == SamplerSnapshotsPerCycle × SamplerPeriodMs (30,000). 1/1.
  • MigrationUpgradeLadderLiveTests (release fixtures v3.3.0 / 3.5.0 / 3.6.0 / 3.7.0 climbed to 133): 5/5. PerfmonCounterTypeLivePostgresTests, DarlingPgDatabaseStatsLiveTests, DarlingAnalysisStoreTests (max stamped version == StorageVersion.SchemaVersion), StoreCopyPhaseLivePostgresTests: all green against the migrated store.

Test plan

New — Darling.Tests/PgNumbackendsAndSampledMsRungTests (5 + 1 live): the rung at the top of a dense ladder; two nullable integer ADD COLUMNs in each collector's generator-rendered type, no VIEW / DEFAULT / NOT NULL / UPDATE / INSERT / DELETE / DROP / INDEX / TABLE, neither table in AllPassthroughViews, both generated CREATE TABLEs and COPY lists carry the column LAST, the rung doc's phrases, HypertableCount 71, payload counts 10 and 7; Lite stores neither table; the probe's three sites + top arm (all-true → StorageVersion.SchemaVersion, one behind → 132, arm order, neither table named in the arm's prose); no reader SQL names either column and both readers' docs say what a consumer must do. PgNumbackendsAndSampledMsLivePostgresTests (gated, above).

Changed — Darling.Tests/PerfmonCounterTypeRungTests hands its top-rung claims to V133's file (the standing handoff idiom; the ordinal-107 pin becomes "fixed, not last"). Darling.Tests/PgWaitSamplerLiveTests reads and asserts sampled_ms. Lite.Tests/PgDatabaseStatsCollectorDefinitionTests: d.numbackends in the select and selected after stats_reset; 10 payload columns with numbackends Integer LAST; the populated-row fixture reads 37, the shared-relations fixture reads NULL as null; the two Row constructions carry the tenth value; WritesEveryPayloadColumnInOrder asserts slot 9; new WritesANullNumbackendsAsNull. Lite.Tests/PgWaitSamplerArmTests: three snapshots → every row SampledMs 3,000 (the 100%-vs-1% comment); the carried-tally cycle → 1,000 on every re-emitted row (per-row, not cumulative); the extension arm → null. Lite.Tests/PgWaitSamplingCollectorDefinitionTests: new TheObservedWindow_IsTheLastPayloadColumn_AndTheExtensionQueryDoesNotSelectIt (7 columns, sampled_ms Integer last after backend_count, absent from the extension query).

Executed here — Darling.Tests on the Mac (WindowsDesktop stripped from the runtimeconfig, DARLING_TEST_PG on the rig): PgNumbackendsAndSampledMsRungTests 5/5 + live 1/1, PerfmonCounterTypeRungTests, NotificationRoutesRungTests, MigrationLadderPins, MigrationDataMovingRungCensusPins, PgSchemaGeneratorTests (the ladder-generator diff — the one census that fired during the lane and drove the V83/V96 CREATE edits), PgLogEventsPipelineTests, both DeltaFamilyInterval*RungTests, PgCopyWriterTests, ReadmeDerivedCountPinTests, StoreSqlClockDisciplineTests, AuroraOnlySqlIsGatedTests, DocCommentHygieneTests, RepoFileAdoptionTests, CommentFilterAdoptionTests, RefreshCeilingProvenancePinTests, PgExtensionDependencyContractTests, MigrationUpgradeLadderLiveTests, PgWaitSamplerLiveTests, DarlingMcpPgWaitToolsTests, PgWaitInstrumentDisclosureTests, LiveCleanupConversionRatchetTests — all green; LivePostgresCollectionHygieneTests.EveryClassUsingTheSharedStore_… reds on the Mac with ReflectionTypeLoadException (the WPF platform artefact; the new live class carries [Collection("live-postgres")] and the LiveStoreCleanup finally). Lite through a net10.0 harness over the built Collectors library (AssemblyName Lite.Tests): PgDatabaseStatsCollectorDefinitionTests, PgWaitSamplerArmTests, PgWaitSamplingCollectorDefinitionTests — 55/55.

First executed in CI, stated plainly: the Windows build job is the first execution of every other Lite/Viewer census; nothing here is SQL Server-side.

Builds, -c Debug -p:EnableWindowsTargeting=true: PerformanceMonitor.Collectors, PerformanceMonitor.Darling.Storage, PerformanceMonitor.Darling.Viewer, PerformanceMonitor.Darling.Service, PerformanceMonitorLite, Darling.Tests — 0 warnings; Lite.Tests — dev's one pre-existing xUnit2000 in LogTailOverlapThresholdPinTests.cs:95, untouched.

Which component(s) does this affect?

  • Lite
  • Darling
  • Lite Tests
  • Darling Tests
  • SQL collection scripts
  • Documentation
  • Full Dashboard (deprecated)
  • CLI Installer (deprecated)

Checklist

  • I have read the contributing guide
  • My code builds with zero warnings (dotnet build -c Debug)
  • I have tested my changes against at least one SQL Server version — not applicable: nothing here touches the SQL Server side; the store side is verified against PG18 + TimescaleDB 2.28.1 (fresh ladder, one-rung climb, four release fixtures, the real COPY writer and the real sampler) as above
  • I have not introduced any hardcoded credentials or server names

…led_ms (Darling V133), so the saturation fraction has a numerator and the sampler arm's wait rate has its duty-cycle denominator (partial #3691)

The ONE migration rung of the #3691 v2 wave: two nullable integer columns,
written by their collectors from this rung on and read by nothing yet.

numbackends: pg_stat_database's client-only connection count per database,
a level sampled every minute, universal since 8.x - the honest numerator for
"how full is max_connections", which pg_session_states could not supply
because it captures only sessions holding an old open transaction.

sampled_ms: the milliseconds the service-side wait sampler actually observed
in each collection (snapshots read x period, 30,000 on a full window). The
arm stores profile_period_ms = 1000 and a cumulative tally but watches 30 s
of every 300 s, so samples x period over the interval understates its rate
~10x and nothing in the row said so. The extension arm writes NULL, having
watched the whole interval; NULL means today's arithmetic.

Both nullable, no default, no backfill, appended last; no view to refresh
(the PG collector tables are view-less); V83/V96 CREATE texts carry the
columns for a fresh store per the V101 rule. Lite stores no pg_* table, so
no DuckDB bump. Viewer probe gains the V133 sentinel and top arm; V132's
test hands the top-rung claims to PgNumbackendsAndSampledMsRungTests.
@erikdarlingdata
erikdarlingdata merged commit f7649cd into dev Sep 19, 2026
7 of 8 checks passed
@erikdarlingdata
erikdarlingdata deleted the feat/3691-v133-pg-columns branch September 19, 2026 10:50
This was referenced Sep 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant