Skip to content

A pg_server_config row now says whose setting it is, so a database that overrides work_mem stops being invisible (#3691) - #3882

Merged
erikdarlingdata merged 7 commits into
devfrom
feat/3691-per-database-settings-v138
Sep 22, 2026
Merged

erikdarlingdata merged 7 commits into
devfrom
feat/3691-per-database-settings-v138

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

pg_settings is the RESOLVED view for the collector's own backend, and the collector connects to one database as one role. So a cluster where one database carries ALTER DATABASE … SET work_mem = '256MB', or one role carries ALTER ROLE … SET statement_timeout = 0, stored nothing about it — the override lives in pg_db_role_setting, which nothing in the stack read — and every reader presented the collector's own resolved value as "the server's configuration". The CONFIG_PG_* facts then graded a value that database's sessions never use. That is the worst shape an advisory number can take: confidently right about a value nobody runs.

V138 adds two nullable columns to collect.pg_server_config — database_name and role_name — so a stored setting row says WHOSE setting it is. Server-wide rows leave both NULL, which is every row this table has ever held; an override row carries one, the other, or both (pg_db_role_setting has three shapes, and the collector stores all three). One rung, two nullable columns, no new table, no new hypertable (TimescaleSupport.HypertableCount stays 72 — V136's doc says what a 73rd would cost the compression grid), no DEFAULT, no backfill, no passthrough refresh (pg_server_config has no v_ view, so the V14 frozen-column-list problem cannot arise). No Lite twin: Lite has no PostgreSQL collectors at all.

The collector's read becomes pg_settings UNION ALL pg_db_role_setting, joined LEFT to pg_database and pg_roles (setdatabase = 0 means ALL databases and setrole = 0 means ALL roles — neither matches an oid, and the join's NULL is exactly the NULL this table stores), with setconfig unnested and split on the FIRST = so a value containing = survives. Ten of the fifteen columns are NULL on an override row and that is not laziness: an override is not a pg_settings row, and the catalog holds a name and a text value and nothing else.

Why the reads are the correctness edge, not the rung

Every shipped read of this table is a latest-snapshot, one-row-per-setting-NAME shape. Several load a dictionary keyed by name (throw on duplicate, or silently shadow); one takes a single row by ORDER BY collection_time DESC LIMIT 1 (an override could decide whether the whole trend read publishes I/O times); one partitions a LAG by name alone (every snapshot would manufacture a change from the server's value to the override's and back). So nine reads gained AND database_name IS NULL AND role_name IS NULL on their OUTER row select, and exactly ONE read asks for the rows the other nine exclude.

# Read Arm
1 PgTargetFactCollector.Config.cs:59 FILTERS (:75-76)
2 PgTargetFactCollector.Memory.cs:43 FILTERS (:56-57)
3 PgTargetFactCollector.Blocking.cs:85 FILTERS (:97-98)
4 PgTargetFactCollector.Posture.cs:60 FILTERS (:76-77)
5 PgTargetBaselineProvider.Clock.cs:70 (TimeZone, LIMIT 1) FILTERS (:86-87)
6 DarlingPgServerConfigReader.cs:138 CurrentConfigSql FILTERS (:153-154)
7 DarlingPgServerConfigReader.cs:190 ConfigChangesSql (inside the ordered CTE) FILTERS (:201-202)
8 DarlingPgServerConfigReader.cs:319 OverrideSql (NEW) SELECTS the overrides (:325)
9 DarlingPgLoggingAuditReader.cs:88 NewestSnapshotSql FILTERS (:103-104)
10 DarlingPgTrendReader.cs:852 IoTimingSettingSql FILTERS (:863-864)

The inner MAX(collection_time) subqueries — seven of them — deliberately carry NO predicate: every one is per SERVER, not per name, so an override row cannot move the anchor, and each comment says so. That is why PgServerConfigScopeRungTests.EveryProductReadOfTheConfigTable_… counts OUTER statements by anchor rather than matching every FROM pg_server_config occurrence (19 across the tree, 10 of them reads), and asserts the partition over the TREE: an eleventh reader arriving with neither arm fails the build. That clause is the point of the census — it catches the next reader, not this one.

source is deliberately NOT the discriminator. The collector's override spellings are database / role / database+role, and database collides with a legitimate pg_settings source value on a backend whose own database overrides the setting. A pin asserts no product read uses source for the scope test.

The surface

get_pg_server_config gains a database_overrides section — [{ database_name, role_name, name, setting }] from the same newest snapshot the settings page reads (both statements anchor on MAX(collection_time) for the server rather than threading an instant between them, which could pin a snapshot retention has since dropped). Absent, not empty, when the cluster has none: JsonOptions writes nulls, so [] would read as "we checked and there are none" on a pre-V138 snapshot too, where the truth is that the collector was not reading the catalog yet. It is uncapped where the settings list is paged, because the question it answers is "is my value overridden anywhere" and a cap turns a No into a maybe; pg_db_role_setting holds one row per scope ever given a setting, which on the measured population is zero. Every existing count (settings_returned, non_default_count, non_default_returned) stays a count of the server-wide population, which is what those names have always promised.

Disk free space is struck from the collector line, with its reason

There is no in-engine SQL source for disk free space on either engine. Stock PostgreSQL exposes no free-bytes function (pg_database_size / pg_tablespace_size are USED bytes, already collected), Aurora exposes FreeLocalStorage only through CloudWatch, and the collectors speak SQL only. No proxy was invented. The checklist face is the coordinator's edit; this PR is where the sentence lives.

Boundary extension, argued

ViewerDataService.cs was outside the stated boundary (the brief scoped the Viewer to ViewerServerTab.Postgres.cs / ViewerPostgresTabs.cs, SQL text only) and needed a V138 probe arm anyway. RequiredStoreSchemaVersion is StorageVersion.SchemaVersion, so without a top arm a fully-migrated store maps to 137 and the connect-time gate refuses a store that is current — and the gate is load-bearing here rather than merely invariant, because the Overview tab's configuration grid reads through the shared DarlingPgServerConfigReader, whose server-wide statement now names both columns in its predicate: a viewer pointed at a store below this rung would throw on that grid. Three edits: the probe SQL's column sentinel, MapProbedSchemaVersion gaining param 114 (hasPgServerConfigDatabaseRoleOverrides) with its return 138 arm ahead of V137's, and the GetBoolean(113) call site. V137's comment was reworded to "one rung behind the top" — a doc line this change falsifies is part of the change. The two Viewer surface files carry only the collector-name string and needed nothing; ViewerDataService.Postgres.cs:332 delegates to GetCurrentConfigAsync and inherits arm 6.

Blast radius

Nine reads gained a two-line predicate each; none changed shape, ordering, cap or index use, and none is on the alert path except by inheritance (the fact reads, which are snapshot-anchored behind (server_id, collection_time) on a few hundred rows). On every store today the predicate matches every row, because no override row can exist before this rung. The new OverrideSql is a second cheap read on a tool path only.

What it does NOT do

The CONFIG_PG_* facts stay server-wide in this lane. A per-database fact family needs a database dimension on the fact, a per-database grading policy, and an answer to "which database's work_mem is the server's work_mem" — a later brief, stated in the rung doc. No alert, no Viewer column, no threshold, no knob. It does not resolve role settings against database settings: PostgreSQL does that per session at connect time, and the catalog records only the instruction.

Test plan

All new tests are compile-verified here and first executed in CI (Darling.Tests is net10.0-windows); the live arms run under Darling PostgreSQL tests.

  • NEW PgServerConfigScopeRungTests (8 facts): ladder + rung name + SchemaVersion == 138; the DDL is one ALTER, two ADD COLUMN IF NOT EXISTS, no DEFAULT/NOT NULL/UPDATE/VIEW/INDEX/TABLE, types rendered from PayloadColumns through PgSchemaGenerator.TypeFor; V102's CREATE text carries both columns and matches the generator's tail (the V101 rule); the rung doc's phrases plus HypertableCount == 72, CollectorCatalog.All.Count == 71, PayloadColumns.Count == 15; the collector's UNION with both arms projecting the same 15 aliases (a UNION ALL silently mis-aligns when they drift) and LEFT-not-inner joins and still no WHERE; source carries the scope kind but is not the discriminator, asserted repo-wide; the reader census (ten anchors by name, the seven inner anchors excluded by shape, reads == 10, and the neither-arm clause); the probe's top arm (ordinal 113 derived from the signature, parameter name, all-true → 138, at-this-rung → 138, one-behind → 137, arm order in source, V71 no-names-in-prose).
  • NEW PgServerConfigOverrideLivePostgresTests (2 live facts): migrate → plant three server-wide rows → read the tool → plant one database-scoped and one role-scoped override naming settings the server-wide rows also carry → the server-wide section is unchanged (3 settings, 2 non-default, no duplicate names) and database_overrides holds both in scope order → delete the override rows → the answer is byte-identical to the pre-override read and carries no database_overrides key. Second fact: the shared server-wide statement and GetOverridesAsync against the same snapshot with a both-scoped third row present — the server-wide read returns exactly the three server-wide names, distinct, and the reader's record keeps database+role intact. These are also the parse-analysis proof for OverrideSql, which names V138 columns and therefore requires a migrated rig.
  • QsCaptureModeRouteKnobToastRungTests handed off the top-rung claims (the V136→V137 handoff, one rung on): RungVersion < SchemaVersion, ProbeOrdinal < arity - 1, its arm returns its own literal rather than the build's version, and its live climb applies every rung from 137 up rather than literally one.
  • RepoFileAdoptionTests.s_lfReaders gains PgServerConfigScopeRungTests.cs with its reason (multi-line anchors).

Build: dotnet build Darling/Darling.Tests/Darling.Tests.csproj -c Debug -v q -nologo -p:EnableWindowsTargeting=true → Build succeeded, 0 errors, 6 warnings, all pre-existing (CA1859 in TimescaleSupport.cs, five CS8602/xUnit1013 in files this PR does not touch). Part A built Storage / Collectors / Analysis / Service clean. The Viewer is not buildable on this host; CI is its first execution, which is stated because the probe edit is in it.

PARTIAL for #3691 — do not close. The collector line reads:

Disk free space; per-database size series (#3794, pg_database_size_stats, hourly, 400 d); per-database settings.

This lane lands per-database settings and strikes disk free space with the reason above.

Checklist

  • Follows the code style of the repository
  • Comments explain the reasoning, not the mechanics
  • New tests cover the behaviour
  • CHANGELOG.md untouched (entry proposed in the lane report)

Two nullable columns on collect.pg_server_config - database_name and role_name - so
a stored setting row distinguishes the server-wide pg_settings population (both NULL,
which is every row this table has ever held) from a per-database or per-role override
out of pg_db_role_setting. No new table, no new hypertable (HypertableCount stays 72),
no DEFAULT, no backfill, no passthrough refresh (pg_server_config has no v_ view).
V102's CREATE text carries both for the V101 rule and the fresh-store generator.
No Lite twin: Lite has no PostgreSQL collectors.
…ide pg_settings (#3691)

pg_settings is the RESOLVED view for the collector's own backend, so a database
carrying ALTER DATABASE ... SET work_mem, or a role carrying ALTER ROLE ... SET
statement_timeout, stored nothing and every reader presented the collector's own
resolved value as the server's configuration. The read now UNION ALLs
pg_db_role_setting - unnest(setconfig), split at the first '=' so a value holding
'=' survives, LEFT JOIN to pg_database/pg_roles so setdatabase=0 and setrole=0
become the NULLs that mean 'all' - and the two V138 payload columns carry the
scope. Ten of the fifteen columns are NULL on an override row because an override
is not a pg_settings row: the catalog stores a name and a text value, nothing more.
source says which kind of override it is but is NOT the discriminator - 'database'
is also a legitimate pg_settings source value - the two new columns are.
…xactly one selects them (#3691)

Ten reads of this table are latest-per-setting-name shapes: four fact-collector
snapshots keyed by name, the baseline provider's TimeZone LIMIT 1, the config
reader's paged server-wide read and its LAG change feed, the logging audit's
by-name lookup, and the trend reader's track_io_timing LIMIT 1. An override row
arriving with a duplicate name would shadow the server-wide value, or - in the
LAG feed - manufacture a change from the server's value to the override's on
every snapshot. Each outer row select now carries
AND database_name IS NULL AND role_name IS NULL; every inner MAX(collection_time)
subquery is per SERVER, not per name, so none needs the predicate and none got it.
The TimeZone read is the sharpest of these: a per-database TimeZone override
would have re-clocked every local-time bucket for the whole server silently.

DarlingPgServerConfigReader gains OverrideSql + GetOverridesAsync - the ONE read
that asks for the rows the other nine exclude - as its own statement rather than a
relaxed filter, because an override is keyed by scope plus name and mixing the two
populations onto one page would put two work_mem rows in front of an operator with
only a column to say which is the server's.

The Viewer's connect-time probe gains its V138 arm (boundary extension, SQL text
plus the mapper): RequiredStoreSchemaVersion is StorageVersion.SchemaVersion, so
without it a fully-migrated store maps to 137 and the gate refuses a store that is
current - and the shared reader the Overview grid calls now names both columns.
…er's settings (#3691)

A database_overrides section from the same newest snapshot as the settings list -
scope, setting name, and the text value that scope was given - absent rather than
empty when the cluster has none, the attach pattern, because an empty list on a
pre-V138 snapshot would read as 'we checked' when the truth is the collector was
not reading the catalog yet. Uncapped where the settings list is paged: the
question is 'is my value overridden anywhere' and a cap turns a No into a maybe.
Beside the settings rather than among them - an override is keyed by scope plus
name, and every count on this answer is a count of the server-wide population.
The note says the stored text is what was SET, not a resolved value, and points
at the server-wide row for the unit and default the catalog does not carry.
…lit, and a pin says which (#3691)

The rung pins move the top-rung claims off V137's file onto V138's, the reader census asserts the ten-read partition over the tree rather than over a list, and the live pair proves the override section is absent-not-empty by byte identity.
…ide row carries its snapshot time (the latest-anchored-read census); the live pin follows the tool's default view — three things the unrun harness would have shipped (#3691, V138)
@erikdarlingdata

Copy link
Copy Markdown
Owner Author

Coordinator, post-report: Part B ran out of budget before the harness, so I ran it in-seat against a migrated throwaway store (71 tests incl. both live arms and McpPayloadContractCensusTests). Three things it caught, fixed in 5b75c5fac: (1) database_overrides was a literal property — "database_overrides": null on every page without overrides (JsonOptions writes nulls); now attached only when owed, the whole page string byte-identical otherwise. (2) OverrideSql did not project its anchor — the latest-anchored-read census would have gone red; it now projects collection_time into PgConfigOverrideRow.CollectionTime, and the live pin asserts every override row is from the same snapshot as the page. (3) The live pin expected the page to hold the default-valued setting; the tool's default view is non-defaults only — expectation corrected. Census green locally.

…the census admits the as-of per-server anchor, column aliases are counted in the SELECT list not the FROM, the SQL is judged without its own comment, the tool pin asserts the attach shape, and a comment stops saying the predicate it forbids (#3691, V138)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant