Skip to content

PostgreSQL-target analysis engine: design and v1 build #3542

Closed
Closed
PostgreSQL-target analysis engine: design and v1 build#3542

Description

@erikdarlingdata
erikdarlingdata
opened on Sep 18, 2026

PostgreSQL targets are skipped by the analysis pipeline outright (DarlingWorker's scheduled arm and analyze_now both write the PostgresAnalysisNotApplicable tombstone), so PG servers get alerts but no scored, baseline-aware, story-built, advice-bearing analysis — the surface the product treats as THE thing an operator's agent talks to. A full design doc was produced during the 2026-09 brains-review campaign; this issue tracks the build.

Shape (from the design doc):

  • The 27-collector PG surface already feeds everything a v1 needs; the only structural blockers are the worker skip and the wait_stats-based 24h data-span gate.
  • Machinery reuses whole: pipeline, scorer formulas, 168-bucket robust baselines, AnomalyGate, inference/clustering, persist/mute/notify, the existing MCP tools (zero new tools). Net-new is CONTENT: fact collectors, scorer arms, thresholds, advice blocks, PG graph edges — in a separate PgTargetFactCollector family (NOT extending PgFactCollector, which analyzes SQL Server targets and is pinned by the Lite-parity census; the PG family gets a self-pin census since Lite doesn't monitor PG).
  • ~10 of the 31 SQL Server fact families map directly (with named traps: PG waits/blocking are SAMPLED sightings, not engine events; Aurora vs stock wait sources are platform-exclusive and carry different evidence grades; deadlock RATE comes from pg_database_stats counters, not the lossy log parser). SQL-Server idioms (tempdb, memory clerks, scheduler, trace flags) are replaced by 11 PG-only detectors: autovacuum backlog/slope, wraparound, xmin horizon, connection saturation, checkpoint pressure, buffer-cache composite, temp-file spills, replication lag + slots, bloat trend, idle-in-transaction.
  • Config surface follows the field-study's two-knob result (75-95% of tuning value from shared_buffers + max_wal_size set per docs): those two amplified only by their workload co-fires, a short convention-check tier, composition checks, and fsync / synchronous_commit / full_page_writes isolated in a pg_posture source no amplifier can touch (durability is policy, never a performance win).
  • Known gap: host RAM is not collected for PG targets, which blocks the memory-overcommit composition checks — the same hardware-profile dimension the campaign flagged as the cross-server transfer blocker. Same collector fix serves both.

Ten maintainer decisions are enumerated in the design doc; the expensive-to-reverse one is pg_-prefixed fact keys/sources (persisted into findings and mutes from day one) and should be settled before the build starts.

Ranked build order: v1 (~11 facts/detectors: the two knobs + connection saturation + the vacuum family + wait profile + temp spills + bad-actor + posture) -> v2 (replication, bloat trend, blocking chains, maintenance-shape) -> v3 (plan regression via pg_statement_stats + captured plans, parameter sensitivity, evidence-gated index candidates).

Build venue: crew-scale feature work — plan validation against the design doc, adversarial review before implementation, then staged build.

Activity

  1. erikdarlingdata commented on Sep 18, 2026

    @erikdarlingdata
    erikdarlingdata
    on Sep 18, 2026
    OwnerAuthor

    Design input from the competitive gap analysis (#3605, filed today): the three PostgresAnalysisNotApplicable call sites in DarlingWorker are the exact seams the v1 build replaces; #3605 proposes fact-family-list-first slicing and asks that PG detector thresholds inherit #3538's measured-lineage discipline rather than constants. Also related from the same analysis: #3601 (target-PG log ingestion umbrella), #3602/#3603 (parser families on it), #3604 (service-side wait sampler fallback for stock PG), #3607 (logging-settings audit, sequenced behind #3601). These are data sources the PG detectors could eventually consume; v1 as designed does not depend on them.

  2. erikdarlingdata commented on Sep 18, 2026

    @erikdarlingdata
    erikdarlingdata
    on Sep 18, 2026
    OwnerAuthor

    Validated v1 build plan (plan-validation pass against dev 950fad3a→2df8fde0, 2026-09-18)

    Full document (537 lines, every claim path:line-cited) is filed on the design bus as pm-issue-queue-inbox/design/3542-V1-BUILD-PLAN.md; this comment carries the parts a lane worker or reviewer needs first. Decisions D1–D10 are adopted as the design doc recommends (D2 pg_-prefixed sources/keys included). #3605's three contributions are folded in (§7).

    0. Headline corrections to the design doc (read these first)

    1. D4 as literally worded cannot be built. The doc says "one DarlingAnalysisService taking an engine
      (constructor composes collector/detector/graph — swap those three by engine)". But the service is
      constructed at THREE sites and two of them are per-process singletons shared across every server:

      • Darling/PerformanceMonitor.Darling.Service/DarlingWorker.cs:5671 — fresh per pass (the worker knows the
        engine here);
      • Darling/PerformanceMonitor.Darling.Service/Mcp/DarlingMcpHostService.cs:471 —
        AddSingleton(new DarlingAnalysisService(postgres, planFetcher, _logger)), one instance for all servers;
      • Darling/PerformanceMonitor.Darling.Service/DarlingWebEndpoints.cs:245 — new DarlingAnalysisService(postgres),
        "shared across requests, like the MCP host's singleton".
        The MCP tools resolve a server by NAME (DarlingServerResolver.ResolveOrErrorAsync returns
        (ServerId, ServerName) only — Mcp/DarlingServerResolver.cs:61) and never see an engine. So D4 must be
        realized as per-CALL engine resolution inside the service        : the one service holds BOTH component sets and
        picks per serverId by reading servers.engine_kind (Mcp/DarlingEngineCapability.cs:49-52
        ServerEngineSql; token vocabulary PerformanceMonitor.Collectors/MonitoredEngineKind.cs:45-62).
        PerformanceMonitor.Darling.Analysis already references PerformanceMonitor.Collectors
        (PerformanceMonitor.Darling.Analysis.csproj:26), so MonitoredEngineKind is reachable without a new
        project reference. The spirit of D4 (one service, one pipeline body, one persist/mute/notify path) is
        preserved exactly; only the selection point moves from constructor to call.

    2. Analysis facts divide by the time the collector actually observed, and a window with a hole in it says so, so collector downtime no longer reads as improvement (#3538 A2) #3592 (MERGED 16:19Z as 2df8fde0, i.e. AFTER the 950fad3a worktree was cut) introduces a NEW
      SQL-Server-ism the doc could not know about, and it is load-bearing.       On dev now: CollectFactsAsync step 0 is a
      coverage witness read from v_wait_stats (PgFactCollector.Waits.cs:46-49 CoverageSql, :93 CollectObservedCoverageAsync;
      PgFactCollector.cs:177 call site), it stamps AnalysisContext.Coverage / ObservedDurationMs, and — decisively —
      the service's empty-window branch is now if (facts.Count == 0 || context.ObservedDurationMs <= 0)
      (DarlingAnalysisService.cs:280 on dev). A PG-target collector that does not stamp its own coverage witness makes
      EVERY PG pass return unavailable ("the collector observed none of the window"). The entry points are now
      CollectAndScoreFactsAsync → (Facts, Coverage) (:479) and ComparePeriodsAsync → 4-tuple (:515); ctor moved
      to :168; TotalDataSpanSql to :607. The plumbing lane must ship a PgTargetCoverageSql over pg_database_stats
      (DISTINCT collection_time, same LAG/gap-policy rules) as its own step 0. The C# edge-finishing BuildCoverage is
      private static in PgFactCollector.Waits.cs:139-160 — the plumbing lane makes it internal static (same assembly)
      or hoists it to FactCollectorHelpers; either way it is not copied. The census in PgFactCollectorTests is now 32
      names (:72 CollectObservedCoverageAsync) — untouched by D1. The doc's D3 choice of pg_database_stats as the
      data-span source is the right witness for the same reason (universal, 1-minute, no extension) — one table answers
      both gates, mirroring how v_wait_stats answers both on the SQL Server side.

    3. Extreme, corroborated anomaly findings can now cross the notify floor, so the baseline engine is no longer notification-inert at shipped settings (Fixes #3526) #3584 (MERGED today) added two more SQL-Server-keyed registries the doc's §5b does not list, and one of
      them is load-bearing for whether PG anomalies can ever notify:

      • FactScorer.IsDeviationScoredAnomalyKey (PerformanceMonitor.Analysis/FactScorer.cs:664-671) is a literal
        list of ANOMALY_CPU_SPIKE / READ_LATENCY / WRITE_LATENCY / BATCH_REQUESTS / SESSION_SPIKE / QUERY_DURATION / MEMORY_PRESSURE. IsExtremeAnomaly (:706-735) and ScoreAnomalyFact (:737-836) both route through it.
        A PG anomaly keyed ANOMALY_PG_* (D2) falls to the ratio arms, scores 0 unless it carries ratio metadata,
        and is NEVER released from the tuning-class cap (IsTuningClassKey :654-656 catches every ANOMALY_
        prefix, including PG's) — i.e. the exact notification-inertness Extreme, corroborated anomaly findings can now cross the notify floor, so the baseline engine is no longer notification-inert at shipped settings (Fixes #3526) #3584 just fixed for SQL Server would ship
        for PG on day one.
      • FactScorer.AnomalyAmplifiers / LoadAnomalyAmplifiers (:921-989): the load-family co-fire arm's measured
        confirmer is CPU_SQL_PERCENT >= 80 (:985-989). PG's CPU fact is PG_CPU_PERCENT (instance CPU, Aurora
        only) — the arm is inert for PG and needs a twin (PG_CPU_PERCENT >= 80, Aurora; absent on stock).
      • PerformanceMonitor.Analysis/AnomalyIncidentReconciler.cs:70-78 AnomalyToFamilies maps anomaly keys to
        SQL Server parent keys (CPU_SQL_PERCENT, IO_READ_LATENCY_MS, BLOCKING_EVENTS, DEADLOCKS…). Not in the
        doc. PG anomalies with no entry stay solo cards (tolerable in v1) but ANOMALY_PG_DEADLOCK_RATE → PG_DEADLOCKS
        etc. should be registered when those keys exist.

    4. D7 needs no code. RunAnalysisPassAsync already writes analysis_state cleared (insufficientData:false, null) after any real pass (DarlingWorker.cs:5820-5823) and the honest gate/empty messages otherwise
      (:5808-5818). Deleting the skip arms + tombstone IS the lazy overwrite; the old row stands until the first
      PG pass runs, which stays truthful.

    5. The doc's PgMigrations.cs line citations are all shifted ≈ +94 (e.g. pg_wait_stats DDL :1806 → :1900,
      pg_database_stats :2265 → :2359, pg_server_config :3492 → :3586, pg_wait_sampling :3759 → :3853).
      Content unchanged — V127 (The four naked delta families store the interval their deltas accrued over, so a restart's fabricated zero reads as unknowable instead of 0.00 ms/sec (#3540, V127 / Lite v60) #3595) added interval columns to the four SQL Server delta tables only.

    6. audit_config lies for a PG target today and will still lie after plumbing unless given an arm
      (Mcp/DarlingMcpTools.cs:409-606): it looks up CONFIG_CTFP / CONFIG_MAXDOP / CONFIG_MAX_MEMORY_MB and, finding
      none, returns status = "no_config_data", "The config collector may not have run yet." — false for a PG
      target whose pg_server_config collector runs hourly. Not in the doc's MCP inventory (§5a lists four
      tools; there are five analysis-service consumers).

    5. PR-sized lanes

                ┌──────── #3592 merged 2df8fde0 (prereq, satisfied) ───┐
            │                                                      │
        Lane P  (plumbing, L)  ──────────────────────────────────►│ keystone
            │                                                      │
   ┌────────┼──────────┬───────────┬───────────┬───────────┐       │
   ▼        ▼          ▼           ▼           ▼           ▼       │
 Lane 2   Lane 4     Lane 7      Lane 8     Lane 3*      Lane 5*   │  wave A (parallel)
 knobs    vacuum     bad actor   posture    saturation   waits     │
  (M)      (M)        (M)         (S)        (S–M)        (L)      │
   │                   │                       │           │       │
   └──────►Lane 6◄─────┘                       │           │       │  wave B
          temp (S)                             │           │       │
   │        │           │           │          │           │       │
   └────────┴───────────┴───────────┴──────────┴───────────┘       │
                              ▼                                    │
                          Lane 9  anomalies/baselines (L)          │  wave C (last)

    * Lane 3 depends on lane 2's config KEYS (pre-declared in P) not its file; lane 5's graph edges touch .Write.cs/.Memory.cs/
    .Saturation.cs chain files owned by lanes 2/3 → land 5 after 2 and 3 or accept a three-file rebase. Lane 6 depends on lane 2's
    Database.cs read and lane 7's Queries.cs drill-down.

    Lane Files Pattern Tests to add Exit criterion Size Depends on
    P plumbing §1 step 1 list §2 §4 "new self-pins" + rewrites PG target: analyze_server → empty w/ coverage ≈ 1; analysis_state cleared after first pass; SQL Server byte-identical L #3592 (satisfied)
    2 knobs Config/Write/Buffer/Database collector partials; scorer/advice/graph partials; audit_config projection ScoreConfigFact, ComposeConfigMaxdop, PgDatabaseSql, PgWriteStatsSql, PgIoSql, CurrentConfigSql PgSettingValueTests; co-fire gating pins; Aurora hit-ratio suppression pin; PG<16 eviction-absent pin quiet default target → 2 advisory cards; churning target → checkpoint story w/ measured ratio M P
    3 saturation Sessions partials PgSessionStatesSql ratio bands; redacted-share → permissions advisory saturation story states sessions/(max−reserved) S–M P (keys from 2)
    4 vacuum Vacuum partials; PostgresOutagePredictorThresholds (Common); PostgresAlertEvaluator aliasing PgAutovacuumSql, PgWraparoundSql, PgXminHorizonSql, PostgresAlertInfo per-counter grading constants-agree source pin; persistence counter; slope arithmetic 3 h backlog series → per-table finding w/ ratio + slope; wraparound/xmin stories share alert bars M P
    5 waits Waits partials; edges into 3 chain files PgWaitStatsSql, PgWaitSamplingSql, WaitRateWindowSql LAG fallback sampled arithmetic; is_sampled presence/absence; restart-collection-not-a-sample; per-source denominator Aurora vs stock same key, stock says "estimated" L P; after 2,3 for edges
    6 temp Temp scorer/advice/graph partials; Queries.cs temp drill-down step 2's Database.cs fact; PgTopQueriesSql work_mem 0 without spill, ≥ 0.5 with PG_TEMP_SPILL → CONFIG_PG_WORK_MEM story w/ offending statements S 2, 7
    7 bad actor Queries partials; PgTargetDrillDownCollector.Queries.cs; tool recs ScoreBadActorFact, PgDrillDownCollector.Queries.cs, PgTopQueriesSql tiering; drill-down join to pg_statement_text; queryid-instability prose pin stories get a query leaf M P
    8 posture Posture partials ScoreConfigFact 0.9 band isolation pins (§4) fsync=off → CRITICAL posture card, zero perf words S P
    9 anomalies PgTargetBaselineProvider, PgTargetAnomalyDetector, Anomaly scorer/advice, AnomalyThresholds, MetricNames, keys' AnomalyToFamilies PgBaselineProvider.cs:288,340-365; PgAnomalyDetector per-detector try/catch; AnomalyGate CTE table pins; deviation-key membership; extremity-escape reachability; anchored gate 30-day planted series → z anomalies fold into steps 2/3/5 parents L P, 2, 3, 5

    Nine PRs; four can open the same day P merges. Every lane: base dev, arm squash auto-merge at open, disposition bot findings,
    no CHANGELOG edits, PARTIAL for #3542 until lane 9 (the last) — lane 9's PR title carries "Fixes #3542" only if every checklist item is
    ticked; otherwise the coordinator closes.

    6. Adversarial pass — where the doc is most likely wrong, and the pin that holds each

    A. Units: sampled PG waits presented on the SQL Server "fraction of period" axis. The doc's §2a-1 mapping puts Aurora
    Σ delta_wait_time_us / observed and stock Δsample_count × profile_period_ms / observed under ONE fact key with is_sampled metadata.
    Two ways this embarrasses the product: (i) the stock estimate has a floor — a wait shorter than profile_period (10 ms default) is
    invisible, and a continuous wait quantizes to ±1 period per backend-sample — so the SAME threshold produces different false-negative
    rates on the two flavors, and OtterTune's authors would ask "what is your measurement error?"; (ii) pg_wait_sampling is
    per-BACKEND-sample — 10 backends in Lock:relation for one second = 1,000 samples = "10 s of waiting" — which is correct as
    time-summed-over-tasks (the SQL Server fraction has the same >1.0 property) but the advice must never say "the server spent X% of the
    window waiting". Hold: (1) metadata estimate_resolution_ms = profile_period_ms and is_sampled = 1 on every stock fact — pinned by a
    test asserting BOTH keys present on stock facts and ABSENT on Aurora facts; (2) an advice-text pin that any composed PG wait advice for
    an is_sampled fact contains "estimated from" and never "spent"; (3) a ScoreWaitFact-style guard that a sampled fact whose
    delta_samples < 3 scores 0 (below three samples the estimate has no resolution). Also the NEW trap from step 5: pg_wait_stats has no
    stored interval (pre-V127 shape) — pin that PgWaitStatsSql computes intervals via LAG(collection_time) and excludes a collection
    whose every delta is a first sighting from the sample count (copy the NULLIF(... , 0) fallback shape from WaitRateWindowSql).

    B. The Aurora-vs-stock evidence-grade split is platform-EXCLUSIVE, and the registry may not know which one you are. Aurora waits
    come only from pg_wait_stats (AppliesTo => target.IsAurora), stock only from pg_wait_sampling (extension; "impossible on Aurora").
    The doc routes on flavor. But servers.engine_kind is NULL until a connect stamps it (DarlingEngineCapability.cs:90-93), and the
    MCP path resolves engine from the registry, not the live target. A NULL-kind PG server would take the SQL Server set (correct fallback
    for the ENGINE decision — it is today's behaviour), but inside the PG collector routing waits by kind would silently pick the wrong
    table on a stale row. Hold: the wait partial reads BOTH sources and emits from whichever has rows in the window (they are mutually
    exclusive by construction, so "both non-empty" is itself a finding worth an ERROR log); the engine resolution itself falls back to the
    data: if engine_kind is NULL but pg_database_stats has rows for the server_id and wait_stats has none, take the PG set (pin:
    ResolveEngineAsync_NullKind_WithPgRowsOnly_TakesPgSet). Coverage honesty (D6) then reports "CPU: not collected on this flavor" from
    CollectorEngineCapability.IsCollectedOnEngineKind (PerformanceMonitor.Collectors/CollectorEngineCapability.cs:509) — and for a
    NULL kind says "flavor unknown" rather than guessing, per that class's own rule (:135-146).

    C. Deadlock RATE from pg_database_stats.deadlocks, exemplars from the log parser — and the two can disagree in public. The doc is
    right that the counter is complete and the log tail is lossy (CollectorScheduleDefaults.cs ~14 KB/s note). The embarrassment is the
    converse: the counter counts every deadlock the engine detected, including ones inside pg_catalog-only or system sessions the parser
    never sees, and is per-DATABASE while pg_deadlocks rows are cluster-wide; a card saying "12 deadlocks/hour" with 3 exemplars looks like
    a bug to a reader unless it says why. Also the PG deadlock ALERT counts pg_deadlocks rows with threshold 1 (PostgresAlertEvaluator.cs:182)
    — alert and analysis would count different things. Hold: (1) fact metadata exemplar_count and counter_count both present; advice-text
    pin that the prose contains "the engine counted N; M were captured from the log"; (2) a source pin that PgTargetFactCollector.Database.cs
    never reads pg_deadlocks for the RATE (DoesNotContain("FROM pg_deadlocks") in the rate SQL) and that the drill-down reads only
    pg_deadlocks (never the counter) for exemplars; (3) rate derived with the reset-aware differencing (stats_reset IS DISTINCT FROM LAG),
    pinned by text — a pg_stat_reset() mid-window otherwise reads as a negative delta clamped to 0, i.e. "no deadlocks" during exactly the
    reset-heavy hours a DBA is debugging. Note for D9: the alert's pg_count_threshold and the analysis rate bar are DIFFERENT instruments
    by design (the evaluator's own comment :170-182 argues the surfaces differ); do not "share" that constant — share only the three
    outage-predictor constants the doc names.

    D (bonus). Composition without RAM. §4b's memory-overcommit check is the doctrine's headline [D4] example and host RAM is not
    collected. The doc offers two framings; both can be wrong in a way OtterTune's authors would notice: treating effective_cache_size as
    "available memory" (it is a planner HINT, routinely 50–75% of RAM and often left at the 4 GB default — the check would fire on every
    default install for the wrong reason). v1 should ship framing (b) only — the RAM-free ratio max_connections × work_mem > k × shared_buffers
    — with k stated as a judgment, and NOT ship (a). Hold: a pin that PgTargetAdvice never mentions effective_cache_size as a memory
    figure (the word may appear only in CONFIG_PG_EFFECTIVE_CACHE_SIZE's own convention card). This is v2 content; the pin belongs with it.

    7. Relationship to #3605 (competitive gap analysis, filed 2026-09-18)

    #3605 ("analyze_server for PostgreSQL targets — the inference engine answers 'what is wrong, ranked' for one engine only")
    is the same problem seen from the competitive side; Erik's cross-link comment proposes it becomes a design input / checklist
    item on #3542. Its three contributions and how this plan absorbs them:

    1. "Three PostgresAnalysisNotApplicable call sites in DarlingWorker" as the exact seams. Confirmed by grep on dev
      950fad3a: DarlingWorker.cs:2500 (scheduled arm), :5936 (analyze_now arm), :8206 (the internal const
      declaration). The design doc's "two sites" counted the two WRITE arms; analyze_server for PostgreSQL targets — the inference engine answers 'what is wrong, ranked' for one engine only #3605's three counts the declaration. Both are
      right; the plan deletes all three plus the PostgresAnalysisStateWritten latch (:696, set at :2493-2495 and :5930)
      — so the true seam count the plumbing PR touches in DarlingWorker.cs is six edit sites in four regions (two arms,
      one latch property, one constant, two rationale comments). The seams analyze_server for PostgreSQL targets — the inference engine answers 'what is wrong, ranked' for one engine only #3605 cannot see from the worker are the ones that
      actually decide whether a PG pass produces anything: the wait_stats span gate (DarlingAnalysisService.cs:607), the
      v_wait_stats coverage witness (Analysis facts divide by the time the collector actually observed, and a window with a hole in it says so, so collector downtime no longer reads as improvement (#3538 A2) #3592, §0.2), and FactScorer.ScoreAll's _ => 0.0 source fallthrough.
    2. Fact-family-list-first slicing ("define the PG fact-family list, wire persistence/read plumbing so PG findings flow
      through the SAME store and alert path, ship two or three families end-to-end rather than all shallowly"). Adopted as the
      plan's structure: lane P declares the WHOLE v1 vocabulary up front (PgTargetFactKeys — every source, key, advisory root,
      anomaly parent; the D2 vocabulary made concrete before a row persists) and the whole collect census as stubs; content lanes
      then fill one family each end-to-end (collector → scorer → advice → graph → tests). analyze_server for PostgreSQL targets — the inference engine answers 'what is wrong, ranked' for one engine only #3605's "two or three families first"
      maps to wave A = lanes 2 (knobs), 4 (vacuum), 8 (posture) — the three with the fewest cross-lane edges and the highest
      doctrine weight ([D1], [D3], [D6]) — with 3/5/7 following and 9 last. Persistence/read/notify/MCP reuse is exactly what
      D4 (one service) and "zero new tools" already require; analyze_server for PostgreSQL targets — the inference engine answers 'what is wrong, ranked' for one engine only #3605 asks for nothing there the doc does not.
    3. Scorer-recalibration lineage ("folklore constants are not acceptable inputs"; inherit Analysis-engine campaign: coverage honesty, measured thresholds, config attribution (adversarial review, 11 findings) #3538 A5/A7's method). Adopted
      as the threshold-lineage rule in §1 step 1 and its source pin — every PG bar is measured, engine-defined, or marked
      unmeasured in code AND in fact metadata. This goes further than Analysis-engine campaign: coverage honesty, measured thresholds, config attribution (adversarial review, 11 findings) #3538 A5 (which re-derives existing bars) because PG has
      no bars yet: the rule is that none is born without lineage.

    #3605 points D1–D10 already answer: "SAME findings store and alert path" → D4 + §5a reuse (no new tables, no new tools);
    "PG engine needs its own fact families" → D1 (separate PgTargetFactCollector with self-pin census) + D2 (pg_/PG_*
    namespacing so families cannot collide); "statement regressions, autovacuum debt, bloat trajectory, wait fractions, connection
    saturation, replication risk" → all six are in the doc's §2/§3 catalogue: autovacuum debt = §3.1 (v1), wait fractions = §2a
    (v1, with the sampled/measured split #3605 does not mention), connection saturation = §3.6 (v1), bloat trajectory = §3.4 (v2),
    replication risk = §3.5 (v2), statement regressions = §2b plan regression (v3 — needs pg_plan_capture corroboration, D8
    sign-off for anything index-shaped); "measured thresholds" → D5 (shared bands, PG calibrated INTO them) + D9 (alert constants
    shared for the outage predictors) + the lineage rule above; "runbook honesty" → the plumbing lane's doc edits
    (docs/postgres-first-target-runbook.md:650-652, Darling/README.md:733,735) — the section keeps saying analysis does not
    exist until lane P lands, then stops, in the same PR. Nothing in #3605 contradicts D1–D10. Disposition: checklist item on
    #3542; close #3605 when lane P merges (the seam-and-plumbing half) with a comment pointing at the lane ledger for the families.

    v2+/data-source dependencies (v1 does NOT depend on any of these)

    Issue What it adds Detector it would unblock (and in which slice)
    #3601 log-ingestion umbrella generalized target-log → telemetry pipeline (both transports) beyond today's two families (plans, deadlocks) prerequisite for #3602/#3603; on its own unblocks a PG_CONNECTION_CHURN / FATAL: too many clients event fact — the event-grain evidence for §3.6 saturation's cliff, which v1 can only infer from the session ratio (v2)
    #3602 log_temp_files parser per-EXECUTION spill events with bytes + statement upgrades §3.9 PG_TEMP_SPILL's drill-down from "statement shapes that spill in general" (temp_blks_written) to "this execution spilled 4.2 GB at 03:07"; v1 keeps the counter-based fact (v2 drill-down)
    #3603 log_autovacuum_min_duration parser per-RUN autovacuum cost (duration, pages, I/O, WAL) unblocks §3.2 PG_MAINTENANCE_SHAPE_SHIFT properly (run duration vs its own hour×dow baseline — the [D3] "overrunning" arm), which v1/v2 otherwise approximate from autovacuum worker presence in pg_session_states (v2)
    #3604 service-side wait sampler for stock PG pg_stat_activity polling into the wait tables when pg_wait_sampling is absent fills the stock-without-extension HOLE in §2a step 5: today such a target has NO wait family at all and D6 reports it structurally absent; with #3604 it becomes a third evidence grade (service-sampled, coarser than the extension) — step 5's is_sampled + estimate_resolution_ms design already has the slot for it (v2, no key change)
    #3607 logging-settings audit (readiness facets) per-setting current/remedy facets for log_lock_waits, log_temp_files, log_autovacuum_min_duration, log_checkpoints, log_connections, log_min_duration_statement the collector-side twin of §4a's meta-advisories: v1's CONFIG_PG_TRACK_IO_TIMING / CONFIG_PG_STAT_STATEMENTS_MISSING come from the config snapshot directly; #3607 would let a PG_MONITORING_COVERAGE finding say WHICH log family is dark and why the matching detector (#3602/#3603) is blind — the D6 coverage-honesty story extended from collectors to target logging (v2)

    8. The three open questions, called (veto window open)

    1. audit_config for a PG target → the plumbing lane ships the honest not_collected envelope pointing at the CONFIG_PG_* facts ("refuse what you cannot honor", the payload contract's rule 8). A PG projection inside the same tool changes its contract for every consumer and is a v2 checklist item with its own description change.
    2. servers.engine_kind NULL with PG rows present → no inference from data presence (PostgreSQL targets have no UI on either SKU, and both UIs render SQL Server tabs at them #2530: a NULL makes no claim). NULL keeps today's SQL Server component set; the pass's honest message names the missing engine stamp so an operator can see why, instead of "still collecting".
    3. pg_wait_stats interval column → it is already one of Measurement-layer campaign: the delta honesty contract (11 findings, one keystone) #3540's four still-naked families; it is the next rung CANDIDATE, not filed as a build dependency. Lane 5 ships the LAG fallback in the V127 COALESCE(stored, LAG)-ready shape so the rung, when it lands, is a one-line change to the read.
  3. erikdarlingdata
    added 8 commits that reference this issue on Sep 18, 2026

    PostgreSQL targets enter the analysis pipeline: the pass routes by re…

    effe482

    A PostgreSQL target's durability posture is stated, not inferred: fsy…

    6ddee19

    PG_BAD_ACTOR: one statement's share of the window's execution time be…

    e0041b5

    The two PostgreSQL knobs learn to speak: shared_buffers and max_wal_s…

    7b19082

    The vacuum family of a PostgreSQL analysis pass: a table persistently…

    ddd9b28

    PG_CONNECTION_SATURATION: the harshest PostgreSQL cliff is graded aga…

    0952c4d

    The temp-spill family of a PostgreSQL analysis pass: spilled bytes pe…

    e9ff9f4

    The wait profile of a PostgreSQL analysis pass: type rollups (Lock, L…

    84fc8f1

  4. erikdarlingdata commented on Sep 19, 2026

    @erikdarlingdata
    erikdarlingdata
    on Sep 19, 2026
    OwnerAuthor

    v2 residue (from lane 3 via the between-waves batch, #3688): pg_database_stats has no numbackends — a RUNG decision (a universal one-minute client-only session numerator per database, instead of the exception-capture peaks pg_session_states gives the saturation fact). Recorded here, not built; the v1 saturation ratio stays on pg_session_states totals.

  5. erikdarlingdata
    added 3 commits that reference this issue on Sep 19, 2026

    The PostgreSQL analysis vocabulary absorbs what six content lanes rep…

    340998e

    PostgreSQL targets get their own baselines and anomalies: five clean …

    f887409

    The Aurora-only-surface census accounts for the wait-profile anomaly …

    829773d

  6. erikdarlingdata commented on Sep 19, 2026

    @erikdarlingdata
    erikdarlingdata
    on Sep 19, 2026
    OwnerAuthor

    Closing — v1 landed and the exit criterion is met, demonstrated on a planted store against the real tool methods; v2/v3 and the lanes' residue move to the follow-on issue #3691.

    What shipped (2026-09-18/19, all merged to dev):

    Exit check (read-only, throwaway PostgreSQL 18 + TimescaleDB, dev @ f887409, real DarlingMcpTools.* methods):

    • Quiet target at shipped defaults → status=findings, five CONFIG_PG_* advisories at 0.4 / confidence 0.20, coverage 1.0 over 241 samples, next_tools → get_pg_server_config.
    • Loaded target → 13 findings, coverage 1.0: PG_CHECKPOINT_PRESSURE → CONFIG_PG_MAX_WAL_SIZE 1.30; ANOMALY_PG_TPS 1.30 (60/s, 25σ over 295 samples); ANOMALY_PG_SESSION_SPIKE 1.30; PG_CONNECTION_SATURATION → PG_WAIT_LOCK_RELATION 1.25 (93/97 = 0.959); PG_BAD_ACTOR_7001 1.25 (60% share, drill-down text); PG_AUTOVACUUM_BACKLOG → CONFIG_PG_MAINT_WORK_MEM 0.94; PG_TEMP_SPILL → CONFIG_PG_WORK_MEM 0.89 (drill-down names the spilling statement); PG_WAIT_IO_DATAFILEREAD 0.71 ("estimated from sampling"); every finding carries advice in PostgreSQL nouns and PostgreSQL-only next_tools (14 distinct get_pg_*, zero SQL Server tools); anomaly and saturation share one incident.
    • get_analysis_facts: 28 facts, threshold_lineage = 0 on every scored PostgreSQL bar and absent on the engine-defined ones; no fabricated zeros.
    • Persists (18 analysis_findings rows, all PG_* / ANOMALY_PG_* / CONFIG_PG_*, confidence_basis populated), mutes (matched_now = 1, the story absent on re-run), occurs (13 → 25), compare_analysis and audit_config answer honestly.
    • SQL Server pass byte-identical to the pre-wave binary (1187a36e) on the same planted store — analyze_server, get_analysis_facts, compare_analysis, cmp equal after timestamp normalisation.
    • Notify: max severity 1.30 < 1.5 on a stock target — by design (the second load-family confirmer is Aurora CPU); the 25σ extremity escape held.

    What v1 does not claim: every PostgreSQL bar is unmeasured tonight and says so on its face — the calibration pass over the dogfood PostgreSQL fleet is the first v2 item; compare_analysis does not yet sigma-band PG_TPS (exit check W1); stock sampled waits have no anomaly baseline; the notify floor on a stock target needs the self-hosted CPU proxy. All on #3691 with path:line.

  7. erikdarlingdata
    closed this as completedon Sep 19, 2026
  8. erikdarlingdata
    added 2 commits that reference this issue on Sep 19, 2026

    CHANGELOG: second splice - sixty-three entries from sixty-five merged…

    00ff250

    audit_config answers a PostgreSQL target with its own settings: the C…

    4db5c04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions