Continuous-aggregate materializations are chunked at one raw chunk, like every raw table, so retention drops what it was designed to and compression reaches the newest day instead of stopping ten days short (#3620)

[erikdarlingdata]

The default nobody set, with the numbers

TimescaleDB sizes a continuous aggregate's materialization chunks at ten times the raw hypertable's chunk interval. Every raw table in the store is created at ChunkIntervalDays = 1, so every one of the twenty materializations carried 10-day chunks — nothing in TimescaleSupport ever set them. On the largest production store the discriminator read confirmed it on all twenty: the 7-day query_store_stats_interval_hourly in two chunks spanning Sep 4 → Sep 24, the 90-day tiers in four spanning forty days.

Two things the product had been reasoning about as if the chunks were a day wide:

1. The interval-identity tiers over-hold. drop_chunks removes a chunk only once its whole range is past the horizon, so the 7-day tier holds between 7 and 17 days at any moment (~14 on the read) and the 10-day tier between 10 and 20. Its measured 71.5 GiB was per ~14 days, not per 7; the per-day ingest derived from it was ~2× high.
2. The continuous aggregates are 235 GiB and none are compressed: the rollups outweigh their 9x-compressed source, and they were never on the compression ladder #3581's compression reaches less than projected. A policy compresses a chunk only once its whole range is past compress_after, so with a 10-day head chunk the newest ~10 days of every aggregate stay uncompressed whatever the window — ~11% of a 90-day tier, most of a 7-day one. Modest night-0 movement on the shipped schedule is the correct expectation, not evidence against The twenty continuous aggregates join the compression ladder: compress_after derived from each tier's refresh window, a once-a-day band off the hourly grid, and the backlog staged one aggregate per night, largest first (#3581) #3611.

What ships

TimescaleSupport.EnsureMaterializationChunkIntervalAsync, called first thing inside EnsureAggregateCompressionAsync (so the worker's order is: aggregates exist → width set → compression enabled and scheduled; no DarlingWorker change). One catalog read over timescaledb_information.dimensions joined to continuous_aggregates on the materialization identity, then per aggregate — under its own try — set_chunk_time_interval(format('%I.%I', materialization_hypertable_schema, materialization_hypertable_name)::regclass, INTERVAL '<MaterializationChunkInterval>'), skipped when the dimension already reads the target. A settled store issues zero statements. The summary line is the #1958 shape: 20/20 materializations chunked at 1 days (one raw chunk, ChunkIntervalDays), N changed this start.

Derivation, not a literal: MaterializationChunkIntervalSpan = TimeSpan.FromDays(ChunkIntervalDays), rendered through the same WholeDaysInterval the compression windows use; the ungated pin asserts the raw tables' CREATE and the materializations' SET carry the identical literal, so the two tiers cannot drift apart. The view name from the registry is the only identity the code carries — the internal _materialized_hypertable_N name is resolved in the statement, and a view the registry does not know is refused.

Why one raw chunk: it is the granularity every raw table already has, the granularity retention and compression both act at (chunks close and eligibility flips at UTC midnight), and the unit #3581's whole compress_after derivation is stated in. It does not touch the disjointness argument #3581 pinned — a chunk compresses only when its whole range is past compress_after, which holds at any width — and the daily tier's 3-day start_offset reads hourly rows by bucket, not by chunk. Cost: ten times the materialization chunks (a few hundred rather than a few dozen, inside what ~70 raw hypertables already carry).

What the rig measured (PG 18 / TimescaleDB 2.28.1)

question	answer
first-level aggregate over a 1-day / 2-day / 3-day raw	10 / 20 / 30-day materialization chunks — 10× the raw's interval as it stands at creation
hierarchical aggregate: root or parent?	Parent, at 1×. A daily built on an hourly set to 5 days got 5 days (not 50, not root×10 = 10); on a parent at 1 day, 1 day. On an untouched store parent = root×10, which is why the #3581 rig read it as root. Consequence: all twenty must be set, and the creation sweep builds all twenty before this runs, so on a fresh store the dailies inherit 10 days too.
set_chunk_time_interval on the materialization via the catalog's two-part name	works; dimensions.time_interval 10 days → 1 day
idempotent	re-run with the same interval: no-op, catalog unchanged
existing chunks	untouched (both 10-day chunks kept their range); a refresh past the old range created 1-day chunks
refresh / compression / retention jobs on the aggregate	byte-identical before and after (config, schedule_interval); compression_enabled unchanged
the full store flow (fresh database, exact live-test call sequence: MigrateAsync → TryEnableAsync → ConvertToHypertablesAsync → ConvergeContinuousAggregateRefreshAsync → EnsureContinuousAggregatesAsync → EnsureAggregateCompressionAsync ×2)	BEFORE 10d ×20 → AFTER 1d ×20; first pass "20 changed this start"; direct re-run returns 0 and says "0 changed this start"; second compression ensure "0 added, 0 converged"; 20 refresh policies and every compression policy in place

The side-effect, stated honestly

Convergence is gradual. Existing 10-day chunks keep their range and age out on their own schedule; only chunks created after the first start on this build are a day wide. The interval tiers' visible hold therefore tightens toward the designed 7 / 10 days over roughly two weeks — a one-time ~5–10 GiB reclaim on the largest store, spread over that fortnight, not overnight — and compression reaches everything older than compress_after + 1 day instead of + 10 on the same timetable. The runbook says so in one sentence, so an operator watching the short rollups' row counts shrink reads it as convergence rather than loss.

Tests

TimescaleAggregateCompressionTests: one new ungated pin (the span, the literal and the statement all derive from ChunkIntervalDays; raw CREATE and materialization SET share the literal; the write resolves the materialization from the catalog and never carries the internal name; refusals for a raw table and an unknown view; the read's join and Time-dimension filter). The existing DARLING_TEST_PG live test gains: the widths read from the catalog before the ensure, every registered materialization at one raw chunk after it, the summary line's change count equal to what was actually wide, a direct re-run returning 0 with its own "0 changed" line and byte-equal widths, and the settled second pass's line carrying "0 changed this start". Cleanup is the existing LiveCleanupBatch.DropContinuousAggregatesAsync snapshot/diff — the width lives on the materialization and goes with it, so no new residue (#1873).

Tests cannot execute on the authoring machine (net10.0-windows); CI is their first execution. Every value they assert was executed against the built assembly and a fresh 2.28.1 database through the same call sequence (table above). Storage, Service and Darling.Tests build at 0 warnings with -c Release -p:EnableWindowsTargeting=true.

What this does NOT do

• No existing chunk is rewritten or split. Nothing forces the old 10-day chunks out early.
• No policy changes. compress_after, the band, the staging, retention horizons and refresh windows are untouched; the disjointness pins stand.
• No schema rung, no DarlingWorker change, no store-metrics changes (DarlingStoreMetricsReader / DarlingMcpStoreMetricsTools / StoreSelfMetrics are other lanes').
• The raw tier's width is not revisited — this makes the materializations follow it.

Docs: one sentence each in Darling/README.md's TimescaleDB section and docs/retention-hold-runbook.md's job-family list. CHANGELOG.md deliberately untouched; the entry is in the lane report.

Closes #3620

[claude]

LGTM — reviewed TimescaleSupport.cs's new EnsureMaterializationChunkIntervalAsync/SetMaterializationChunkIntervalSql/MaterializationChunkIntervalStateSql, the test coverage, and the README/runbook doc updates. Correctness: ordering (called first inside EnsureAggregateCompressionAsync, after EnsureContinuousAggregatesAsync at the DarlingWorker call site), per-aggregate failure isolation, and the settled-store no-op path all check out; the derivation of MaterializationChunkIntervalSpan/MaterializationChunkInterval from ChunkIntervalDays keeps the raw and materialization widths from drifting, and the pinned test asserts the identical literal appears in both CreateHypertableSql and the SET statement. Security: the interpolated view name in SetMaterializationChunkIntervalSql is never user input — AggregateCompressionTargetFor(view) validates it against the fixed static registry (AggregateCompressionTargets) and throws ArgumentOutOfRangeException for anything else, so there's no injection surface. Style matches existing conventions in this file (AS aliases, uppercase keywords, ON/AND alignment). Parity: this is TimescaleDB-internal chunk tuning with no DuckDB analog, and Lite/README.md has no continuous-aggregate content to begin with, so no Lite counterpart is expected here (consistent with #3581 before it). No blocking findings.