Skip to content

A PostgreSQL table that turned autovacuum off and fell behind is its own card: CONFIG_PG_AUTOVACUUM_DISABLED names the table, how far past its own trigger line it sits and for how long, and joins the backlog's story on the same table (#3691 step 22) - #3761

Merged
erikdarlingdata merged 1 commit into
devfrom
feat/3691-pg-autovacuum-disabled
Sep 19, 2026

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

What does this PR do?

Headline mechanism. A PostgreSQL target's analysis pass now emits CONFIG_PG_AUTOVACUUM_DISABLED — design §3.1's "per-table autovacuum_disabled = true with a backlog is its own 0.9-band fact" — for the worst table (by ratio to its OWN trigger line) whose autovacuum_enabled reloption is off and which has sat past that line for BacklogPersistenceSamples (3, measured) consecutive hourly pg_autovacuum_stats samples. A disabled table with no backlog is not a finding (someone may be vacuuming it by hand); an enabled backlogged table is PG_AUTOVACUUM_BACKLOG's business. What a PostgreSQL operator now learns, value-stated: "public.frozen in appdb has autovacuum_enabled = off and sits at 4× its own autovacuum trigger line for 6 hours" — with the dead tuples against the table's own threshold (or the insert arm's figures), how long since autovacuum last ran (or that it never has), the table's size, whether autovacuum is ALSO off server-wide, and the shape of the other disabled tables that met the gate ("2 more disabled tables have met the same gate (3.1× for 5 hours, 1.4× for 2 hours) — get_pg_autovacuum_health names them").

v1 (lane 4, #3674) folded the reloption into the backlog fact's metadata + a 0.5 amplifier + a remediation sentence. This lifts it into its own fact — the named CAUSE beside the backlog, which keeps the grade. The v1 amplifier stays: the backlog IS worse for being disabled, and that boost is what makes the backlog lead the story (see the chain below).

The key and its routing. One new constant, PgTargetFactKeys.ConfigAutovacuumDisabled, in the lane-4 block. It is CONFIG_PG_-prefixed because the finding IS a configuration (one the operator set with ALTER TABLE … SET (autovacuum_enabled = off)), but its source is pg_vacuum, not pg_config: the value is per table, read from pg_autovacuum_stats, never from pg_settings. Consequences, each stated on the key's doc:

  • Base score routes by SOURCE → ScoreVacuumFact gets a new arm (ScoreAutovacuumDisabled), in the vacuum partial. Flat AutovacuumDisabledBaseSeverity = 0.9 once the two gates hold (persistence ≥ BacklogPersistenceSamples by name, ratio ≥ 1.0); no ramp, because the ratio is graded on the backlog fact for the same table and a second ramp would double-count it.
  • Amplifiers route by KEY → the CONFIG_PG_ prefix arm → ConfigAmplifiers → []. Deliberate and documented in the scorer header: the co-fire D5 would add is the condition of the fact's emission, and the backlog already carries the +0.5 reloption boost. Two boosts on one reloption would count it twice.
  • Advice routes by KEY → the prefix arm → ComposeConfig → null. That would leave a story rooted on the card with no advice, so the root dispatcher PgTargetAdvice.Compose names the key in the vacuum arm — the exact shape ConfigAutovacuumOff / ConfigMaintWorkMem already take there. This is the one edit outside the STEP brief's file list (one token, one comment, in PgTargetAdvice.cs); the routing census (EveryDeclaredPgKey_IsRoutedByEveryRootDispatcher) treats named config keys as a subset and is green.
  • Not in ConfigAdvisoryRoots: it clears the incident line on its own evidence, so D5's any-positive-severity root rule is not needed and would be wrong (it is not a convention check).

The read. PgTargetAutovacuumDisabledSql is the backlog read's CTE chain and projection with a different tail: WHERE l.autovacuum_disabled AND l.trailing_samples_past_line >= $4, ranked by ratio (every row is disabled, so no disabled-first ORDER), LIMIT $5 = 3. To make "the same runs" structural rather than a promise, the ninety lines of window arithmetic are now two private consts (BacklogRunsCte, BacklogProjection) and BOTH public *Sql consts are halves + own tail — compile-time constant concatenation, so AllSql's reflection closure and the dialect / FROM-target censuses read the finished text. PgTargetAutovacuumBacklogSql is byte-identical to dev (a pin compares the two consts up to their final WHERE, and the v1 live pins on the backlog read execute unchanged). $4 is bound from PgTargetScorer.BacklogPersistenceSamples in both reads, so the two facts cannot disagree about whether a table is backlogged. Own degrade around the read (same table, same 42P01 classification, abandonment not swallowed). Whether autovacuum is also off server-wide is read off the CONFIG_PG_AUTOVACUUM_OFF fact the config read emitted earlier in the pass (emission order Config → Vacuum) — one pass, one truth about the setting, no second pg_server_config read.

Metadata. The worst table's figures ride under the backlog metadata keys (one vocabulary; the advice reads both facts with the same names): ratio, arm, trailing samples, samples in window, dead/threshold/live, inserts/insert-threshold, total bytes, backlog_hours (the run's own span — latest sample minus the run's first, never an assumed cadence), hours_since_last_autovacuum when present. New: disabled_tables_in_backlog (count that met the gate), server_autovacuum_off (0/1), and disabled_rank_{2,3}_ratio / _hours for the shape of the next two. Fact.Metadata is doubles, so names beyond the worst cannot ride; the advice states the shape and sends the reader to get_pg_autovacuum_health (disabled tables first) — the same convention the backlog fact and ANOMALY_OBJECT_GROWTH already follow. threshold_lineage = 1.

Lineage of every bar (PgTargetThresholdLineageTests executed green)

Bar Shape
AutovacuumDisabledBaseSeverity = 0.9 engine-defined — the design's §3.1 band for "the engine's own maintenance was switched off on an object that needs it": D5's advisory base raised into the incident band because the workload co-fire is built into the emission (the fact exists only for a table past the line PostgreSQL itself draws). Below 1.0 because the grade lives on the backlog. Stated on the constant.
Ratio ≥ 1.0 (the table's own trigger line) engine-defined
Persistence = BacklogPersistenceSamples (3), by name measured (#3723, 2026-09-19) — reused, never retyped

No new boost, no absolute-ms gate, no SQL Server constant.

Co-fire gating and the chain

Graph (BuildVacuumEdges, four edges): PG_AUTOVACUUM_BACKLOG ↔ CONFIG_PG_AUTOVACUUM_DISABLED, gated on the two facts naming the SAME table (DatabaseName and schema.table, ordinal, both present — SameTable); CONFIG_PG_AUTOVACUUM_OFF ↔ CONFIG_PG_AUTOVACUUM_DISABLED (the launcher being off server-wide subsumes every per-table reloption). Both directions each, for lane 4's mesh reason: the walk roots at the highest amplified severity and follows edges outward, so a one-way edge orphans whichever end does not lead. Executed shapes, all pinned:

  • backlog (4.2× = 0.68, × 1.5 reloption boost = 1.02) + card (0.9), same table → one story PG_AUTOVACUUM_BACKLOG → CONFIG_PG_AUTOVACUUM_DISABLED, the card the leaf.
  • same, different table (or same schema.table in a different database) → two findings, and the edge is inactive.
    • autovacuum = off server-wide → backlog 0.68 × (1 + 0.5 + 0.5) = 1.356 leads autovacuum = off at 0.9 × 1.5 = 1.35 by a hair → one story of three, BACKLOG → OFF → DISABLED, the last hop the new edge. The order is pinned because the mesh's promise is that the path follows severity.
  • Live: planted frozen (disabled, 4× for 7 hourly samples spanning 6 h, never vacuumed), quiet (disabled, under its line), hot (enabled, 2×) → exactly ONE CONFIG_PG_AUTOVACUUM_DISABLED naming frozen; quiet appears nowhere; the backlog fact names frozen too (disabled-first ranking) and counts hot beside it; through the REAL analyze_server: findings, one story PG_AUTOVACUUM_BACKLOG → CONFIG_PG_AUTOVACUUM_DISABLED, fact_count 2, no second root on the card, next_tools get_pg_* only; get_analysis_facts source=pg_vacuum shows the two with threshold_lineage = 1 on the card.

The graph doc's "config leaves are one-directional (a setting is never a root of this chain)" was falsified by a config key that CAN root; the paragraph now says which two are one-directional and why this one is not.

Advice (OtterTune doctrine)

Every number read from the fact; two levers, each with its counter-objective: (1) re-enable — ALTER TABLE public.frozen SET (autovacuum_enabled = true); (no restart; the launcher vacuums it at once because it is already past its line) at the cost of the vacuum I/O whoever disabled it was avoiding, with the per-table autovacuum_vacuum_cost_delay reloption named as the throttle that does not require leaving it off; (2) keep it off and schedule VACUUM (ANALYZE) public.frozen; yourself, with the cost stated (every page of the table's N MB read, SHARE UPDATE EXCLUSIVE held — reads and writes continue, DDL and other vacuums wait — and a schedule that slips becomes this finding again). Never VACUUM FULL as the first lever (ACCESS EXCLUSIVE, full rewrite). Dead-tuple arm adds the lower-line variant (autovacuum_vacuum_scale_factor = 0.053, half the table's current effective fraction, computed). Server-off case leads with "turn autovacuum back on server-wide first". Static block for the read-time fallback. The regex pin that no vacuum block recommends disabling autovacuum holds on the new block; no CREATE INDEX (D8); statements in prose, RemediationTsql null. next_tools: get_pg_autovacuum_health, get_pg_table_bloat.

What it does NOT do

  • No names beyond the worst table in the fact (Fact.Metadata is doubles) — the shape of ranks 2–3 rides, the names are the tool's.
  • No amplifier on the card (above), no change to the backlog's existing reloption amplifier or advice sentence.
  • No DrillDown list of disabled tables on the finding (a PgTargetDrillDownCollector matter, out of lane).
  • No schema, no knob, no migration; the SQL Server pass is untouched (every edit is inside PG-only partials plus one token in the PG advice root dispatcher).

Blast radius

PgTargetAutovacuumBacklogSql byte-identical (pinned). One new read per PostgreSQL pass over a table the pass already reads, behind its own degrade. PgTargetAdvice.cs: one token in an existing arm. Every other change is inside the four lane-4 vacuum partials, the keys file (+1 const), the tool-recommendations table (+1 row) and the two vacuum test files. The v1 vacuum live test still asserts total_facts 4 / shown 3 because none of its tables is disabled.

Test plan

New (compile-verified AND executed here through a net10.0 xunit harness inside the worktree — AssemblyName=Darling.Tests — against a throwaway timescale/timescaledb:2.28.1-pg18 on Docker context desktop-linux (superuser darling, migrated to V132); 432/433 green across the 35-class harness set, the one red being the harness's own folder tripping DocCommentHygieneTests.TheDerivedProjectListCoversEveryProjectInTheTree (walk-up artifact, not in the diff); first CI execution is the Windows job):

  • PgTargetVacuumTests (+6, now 62): flat band past both gates / 0 under either / lineage 1 / no amplifier results; the disabled read shares the backlog read's CTE+projection byte-for-byte and adds only the reloption filter (+ backlog read keeps disabled-first ORDER, disabled read has none, AllSql membership); same-table one-story vs different-table two-stories vs different-database edge inactive; server-off three-fact story with the pinned severity order; advice value pins (headline, dead-tuple arm figures, hours, last-autovacuum, size, shape of ranks 2–3, both levers, three counter-objectives, 0.053, insert arm names its own line, never-disable regex, next_tools); the static-block census gains the key.
  • PgTargetVacuumLiveTests (+1 gated [Fact], same class/collection/cleanup): the exit criterion above, executed green here.

Pre-checked rosters (executed green here): PgTargetSharedSwitchRoutingTests (routing census with the new key, delegation equality), PgTargetThresholdLineageTests, PgTargetFactCollectorTests (AllSql closure — every NpgsqlCommand site names a public *Sql const; dialect; FROM/JOIN targets; e2e), PgTargetMcpSurfaceTests, PgTargetPostureIsolationTests, StoreSqlClockDisciplineTests, DocCommentHygieneTests (all but the harness-folder artifact), FactCollectorCommandTimeoutTests, FactCollectorFailureReportingTests, AnalysisPassCommandTimeoutTests, AnalysisPassTokenThreadingTests, FactSourceRegistryTests, PgTargetKnobsTests, PgTargetBetweenWavesTests, PgTargetSessionsTests, PgTargetTempTests, PgTargetPostureTests, LiveCleanupConversionRatchetTests, RepoFileAdoptionTests (no LF reader added), CommentFilterAdoptionTests, LivePostgresCollectionHygieneTests. No aurora_stat_* mention anywhere in the diff.

Two things executing the pins caught that compile-only would have shipped: (1) FactScorer caps Severity at 2.0, not 1.0 — a first-draft pin assumed the boosted backlog would read exactly 1.0; (2) with autovacuum = off in the set, the backlog leads by 1.356 to 1.35 rather than the server setting leading — the story is still one, the pinned order is the executed one.

Builds, zero warnings: PerformanceMonitor.Analysis, PerformanceMonitor.Darling.Analysis, PerformanceMonitor.Darling.Service, Darling.Tests, Lite.Tests, deprecated/Dashboard.Tests (-p:EnableWindowsTargeting=true where windows-targeted).

Which component(s) does this affect?

  • Lite
  • Darling
  • Lite Tests
  • Darling Tests
  • SQL collection scripts
  • Documentation
  • Full Dashboard (deprecated)
  • CLI Installer (deprecated)

How was this tested?

Above. Throwaway TimescaleDB 2.28.1 / PostgreSQL 18 container on Docker context desktop-linux, torn down. No production store or fleet server was read.

Checklist

  • I have read the contributing guide
  • My code builds with zero warnings (dotnet build -c Debug)
  • I have tested my changes against at least one SQL Server version (no SQL Server path is touched; the change is inside PostgreSQL-only partials and the PG advice dispatcher's PG arm)
  • I have not introduced any hardcoded credentials or server names

Partial for #3691 — completes the checklist line: "Per-table CONFIG_PG_AUTOVACUUM_DISABLED (design §3.1) has no key". CHANGELOG deliberately untouched — the coordinator splices this wave's entries.

…ONFIG_PG_AUTOVACUUM_DISABLED names the table, its ratio to its own line and how long it has sat there, and joins the backlog's story on the same table (#3691 step 22)
@erikdarlingdata
erikdarlingdata merged commit b0a627a into dev Sep 19, 2026
7 of 8 checks passed
@erikdarlingdata
erikdarlingdata deleted the feat/3691-pg-autovacuum-disabled branch September 19, 2026 17:48
erikdarlingdata added a commit that referenced this pull request Sep 19, 2026
… a set, not by count: step 22 gave pg_autovacuum_stats a second vacuum read, and one missing family is one however many reads it serves (#3691) (#3767)

PgTargetCollectionCaveatsLiveTests pinned Assert.Single on the entries
of a blind pg_autovacuum_stats; #3761 (step 22) landed
ReadAutovacuumDisabledAsync beside ReadAutovacuumBacklogAsync in the
same family between this lane's rebase base and its merge, so the CI
merge ref recorded two entries — both vacuum, both missing_schema,
families_failed still 1, exactly as designed. The pin now asserts every
entry is vacuum / missing_schema / names the table, that the backlog
read is among them, and keeps families_failed = 1 of 16 and the
sentence; the facts-read and service-side assertions follow suit.
Executed live against current dev on a throwaway PG18/TimescaleDB.
erikdarlingdata added a commit that referenced this pull request Sep 22, 2026
…three PostgreSQL cards stop hiding table names in metadata keys (#3691) (#3864)

* A summary fact can rank its top few objects by name, so a card stops saying "two more, 3.1x for 5 h" without saying which (#3691)

Fact carried one string per scope (DatabaseName, ObjectName) and a doubles-only
Metadata, so a top-N card could name exactly one object. Three closed lanes each
worked around it differently: #3761 rode ranks 2 and 3 as numbers without names,
#3722 encoded the name into the metadata KEY (growth_bytes_<schema.table>), #3674
capped its read at one row and deferred the list to a tool.

Fact.Ranked is a typed, capped list of RankedObject (ObjectName, DatabaseName,
Value, Figures) whose [0] IS the fact's own subject, so a card has one truth about
what it is about. Cap three (FactRanked.MaxObjects) - the card is a summary, not
the list. Fact is never persisted (only AnalysisFinding is; get_analysis_facts
re-runs the pass), so the seam is ephemeral by construction: no migration rung, no
store column. Metadata stays doubles-only.

FactAdvice.NameTheRest is the one grammar for the clause every migrated family
gains ("; and two more: public.orders (3.1x for 5 hours), ..."), empty below two
ranked objects - which is the byte-identity arm for every fact that ranks nothing,
including every SQL Server fact. Each family passes its own figures formatter
because each ranks in its own units.

* The autovacuum-disabled card names the other disabled tables instead of describing their shape (#3691, #3761)

M1 of the Fact.Ranked migration. The disabled read has returned three rows since
#3761, but only the worst one had a name: ranks 2 and 3 rode as
disabled_rank_2_ratio / disabled_rank_2_hours, doubles in a doubles-only
dictionary, so the best the card could do was "2 more disabled tables have met
the same gate (3.1x for 5 h, 1.4x for 2 h)" - the reader was told the shape of two
problems and had to call a tool to learn which tables they were.

Every returned row now rides on Fact.Ranked with its name, its ratio and its
hours, the worst included as Ranked[0] (the seam's invariant: [0] IS the fact's
ObjectName/Value). The clause lands on the sentence that states the SUBJECT's
ratio and hours, through FactAdvice.NameTheRest, precisely because [0] is the
subject - "and two more" can then only be read as "besides the table just
described", never against the population count that follows. That population
sentence keeps its own number and its own pointer to get_pg_autovacuum_health: the
gate's population can exceed the three rows the read returns, so the named clause
is a subset of it on purpose.

The two rank-key builders are deleted rather than left beside Ranked. Two places
to read the same rank is how a card ends up disagreeing with itself. A row whose
name triple is entirely null is not ranked at all - an unnamed entry would break
the list's own promise. The row limit is now FactRanked.MaxObjects by name, which
is the three this read chose first and the seam then adopted.

Executed on macOS against the real composer before the pins were written.

* The bloat trends' top objects stop hiding their names inside metadata keys (#3691, #3722)

M2 of the Fact.Ranked migration. Since #3722 the two bloat trends have carried
their top three objects as KEY-ENCODED metadata: growth_bytes_<schema.table>,
growth_pct_<...>, dead_tuples_<...> - the object's name inside the key, because
Fact.Metadata is doubles-only - and PgTargetScorer.BloatNamedObjects recovered
those names with a StartsWith test over every key on the fact. A naming convention
doing a type's job, and the graph predicate that reads it had to trust that the
convention held.

The rows now ride on Fact.Ranked, one typed entry per returned row with its name,
its database and its three figures under the fact's own UN-PREFIXED metadata names
(growth_bytes, growth_pct, dead_tuples), the worst as Ranked[0] per the seam's
invariant. The three prefixes are deleted, not kept beside the list: while both
existed an object could be named in one and not the other, and the graph predicate
reading one would disagree with the advice reading the other.

BloatNamedObjects keeps its signature and its semantics exactly - it is a
graph/amplifier predicate helper (PgTargetRelationshipGraph.Bloat, two scorer
amplifiers, the index card's table co-fire) - and only its SOURCE moved, including
the parent-table projection that turns schema.table.index into schema.table. It
still unions Fact.ObjectName in rather than trusting Ranked[0], because hand-built
facts reach it and a fact that names an object without ranking must still match.

The cards' prose is byte-identical: "Next by growth: public.events (300 MB, 40%,
20,000 dead tuples now); sales.ledger (10 MB)." renders the same characters
through the shared NameTheRestList grammar, executed on macOS against the real
composer before the pins moved. A fact with one object over the floor emits no
such sentence at all, exactly as before. The database-growth family's
growth_bytes_<name> encoding (PgTargetAdvice.Growth.cs) is a different family and
is out of this lane; its reader stays where it was, named as a candidate for the
same move.

* [WIP M3] The backlog card reads three rows and ranks two more tables beside the worst (#3691, #3674) — advice prose and pins unfinished

* M3 done: the backlog card's live pins name the tables it ranks (#3691, #3674)

* Both engines' analysis payloads name the objects their cards rank, instead of leaving an agent to parse them out of the prose (#3691)

* The ranked seam's own pins: the invariant every ranking collector owes, the payload's two-or-more rule, and both SKUs' parity (#3691)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant