You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Ingest target PostgreSQL logs as telemetry — errors, spills, autovacuum runs, and connection churn are invisible between counter samples #3601
The PostgreSQL log is the engine's primary event record — errors, cancelled statements, temp-file spills, autovacuum runs, connection churn, and lock-wait reports all land there and nowhere else. Darling currently reads target logs for exactly two families: auto_explain plans (PgPlanCaptureCollector / RdsPlanIngestor) and deadlocks (PgDeadlocksCollector / RdsDeadlockIngestor). Everything else in the log is invisible, and the view-based collectors can only see what survives into a cumulative counter — a burst of FATAL connection failures, a 40-second statement cancelled by a client, or a temp-file storm between two samples leaves either a bare counter increment or nothing.
For the SQL Server DBA audience this is the sharpest remaining gap: their instinct of "check the error log" has no Darling answer today, and pganalyze's guide for that audience (https://pganalyze.com/blog/postgres-monitoring-for-sql-server-dbas) treats log workflows as a primary pillar.
The expensive part already ships. Both transports exist and run in production — pg_read_file tailing for self-hosted targets and the RDS/Aurora log API for managed ones — with cursoring, timezone handling (PgLogTimezoneUnsupportedException), and per-family parser precedent (PgDeadlockLogParser, PgPlanLogParser). What this issue proposes is generalizing that machinery into a log-event pipeline: a classifier that walks the same tailed stream and fans lines into event families (error/severity events first; sibling issues cover temp-file and autovacuum families), a hypertable for classified events with the store's usual retention/compression discipline, and a read surface (get_pg_log_events) with severity/family/time filters.
Scope note: statement literals in error context lines must get the same redaction discipline PgPlanLogParser applies to plan capture — the log pipeline must not become the place parameter values leak into the store.
Sibling increments filed separately (each can ride this pipeline independently, as deadlocks already prove): per-event temp spills, autovacuum run costs, and a logging-settings audit.
Activity
get_pg_logging_audit judges a PostgreSQL target's logging settings fa…
PostgreSQL's log is read by a classifier now, not by two regexes: err…
get_pg_logging_audit judges a PostgreSQL target's logging settings fa…
Review + census: the key-tuple redaction runs to the tuple's true clo…
PostgreSQL's log is read by a classifier now, not by two regexes: err…
Review + census: the key-tuple redaction runs to the tuple's true clo…
PostgreSQL's log is read by a classifier now, not by two regexes: err…
Review + census: the key-tuple redaction runs to the tuple's true clo…
PostgreSQL's log is read by a classifier now, not by two regexes: err…
erikdarlingdata commentedon Sep 18, 2026
Closed by the merge of #3646 (its body names this issue with a closing keyword; keywords do not fire against dev, so the watcher closes it).
CHANGELOG: the 2026-09-18 board - fifty-four entries from two seats, …
CHANGELOG: the 2026-09-18 board - fifty-four entries from two seats, …