Adopt zstd TOAST compression for the store when PostgreSQL ships it

[erikdarlingdata]

Tracking issue. zstd TOAST compression is not in PostgreSQL 18 (verified against the production 18.4 binary: default_toast_compression accepts pglz, lz4 only; the upstream patch is still in progress on -hackers). When a PG release ships it AND TimescaleDB supports that major:

1. Upgrade the bundled pg-runtime + the compose timescale/timescaledb image.
2. Set default_toast_compression = zstd in the managed-PG config (or per-column on the dimension payload columns).
3. Nothing else: the change is transparent to every reader, applies to newly written values, and the plan dimension's ~9-day GC turnover converts the whole table without a rewrite.

Context: measured on live content 2026-08-05 — lz4 (current) 8.9× on plan XML, app-level gzip 14.0× (#2069 ships that interim), zstd expected ~16–20×. When zstd TOAST arrives it should replace #2069's app-level layer entirely (delete the gzip write path + reader decompression, let the engine do it), restoring raw-SQL access to plan XML as a bonus.

[argpna]

Hey Erik, thanks for creating this. Consuming it downstream, I only realized after plan XMLs failed to render after 3.4 upgrade :). Currently working around using CREATE EXTENSION IF NOT EXISTS plpython3u... then public.darling_gunzip(bytea) RETURNS text. That works alright for now...

[erikdarlingdata]

@argpna — thanks for surfacing this; you're the first confirmed downstream store consumer hit by the V54 gzip migration, and that's exactly the kind of contract change that deserved louder release notes. Your plpython3u darling_gunzip workaround is sound. Two notes: (1) the apps and MCP tools decompress client-side, so nothing in-product breaks — the gap is purely for direct SQL consumers like you; (2) rather than everyone hand-rolling UDFs, the store should probably ship a documented decompress helper (or a compatibility view exposing decompressed XML) as part of the schema itself — I'll fold that into this issue's scope since zstd TOAST would hit the same consumers again. If you can share what you're building downstream, it helps shape the contract.

[erikdarlingdata]

Correcting my scope call from earlier: I said I'd fold the decompress-helper contract into this issue, and that was wrong — this issue is version-gated on PostgreSQL shipping zstd TOAST, while @argpna's breakage is live today. Split out to #2171, which ships in the 3.5.0 window. This issue stays parked on the upstream dependency, and #2171's contract is what protects consumers when zstd eventually lands here.

[argpna]

replied here - #2171 (comment)

[erikdarlingdata]

Cross-linking the resolution path: @argpna confirmed on #2171 that their consumer reads the store from SQL only (Grafana straight off darling-pg), so #2171 will ship a plan_xml_compression store setting whose none value writes plain query_plan_xml and lets lz4 TOAST do the compressing.

Re-measured against this issue's 2026-08-05 baseline on the production store today and it holds: app-level gzip 14.2x (was 14.0x), 20,000 plans sampled, 189 MB stored vs 2,690 MB raw. Against lz4's 8.9x that means engine-native costs 1.6x on plan storage — the plan dimension 100 GB → ~160 GB — not the 14x the gzip ratio suggests, because gzip'd bytea is incompressible and gets nothing from TOAST today.

That makes #2171's setting a partial early landing of this issue's endgame: SQL-readable plan XML via engine compression, available now at lz4 rather than blocked on zstd. When zstd TOAST ships here, both values of the setting converge on engine compression and the app-level gzip path deletes as planned above.