Repository navigation
Compression policies still drift finish-to-start; pin them to the phase grid to finish #3012's deconfliction #3035
Description
#3012's fix (PR #3024) pins the hourly continuous-aggregate refresh policies to a fixed 15-minute phase grid, which ends their finish-to-start drift. Compression policies are the other half of the convoy pairing that issue documents, and they remain unpinned: add_compression_policy is called with a schedule_interval but no initial_start, so every compression job keeps TimescaleDB's finish-to-start scheduling and drifts through the hour by its own runtime each cycle.
Why this is a real residual and not a theoretical one
- The Collector store-writes are cancelled by the store's own CAGG refresh and compression: a lock convoy on query_store_stats #3012 convoy head was compression queuing an
AccessExclusiveLockbehind a long-running refresh'sAccessShareLockon the same hypertable, convoying every collector store-write behind it. - On the production store measured 2026-09-05, all compression jobs report
fixed_schedule = f. The one hand-staggered compression job holds near its hand-set minute only because its runs are typically seconds long — drift is proportional to runtime, and a run that finds a day's chunk eligible takes minutes, not seconds. - A same-day live read caught the pairing one coincidence away from re-forming: a compression job fired inside the drifted heavy-refresh window and missed a convoy only because that particular run had nothing eligible (0 s) and that particular refresh ran light (194 s versus its 864 s measured heavy cycle). Two favourable rolls, neither guaranteed.
- With refreshes pinned by Narrow the hourly refresh window and phase the refresh family across the hour (#3012) #3024 and compression drifting, the overlap recurs on a slow lap: each compression job wanders the hour and periodically crosses each fixed refresh slot, spending hours-to-days inside a window before drifting out.
Proposed shape (mirror #3024's mechanism)
- A compression slot on the same grid, clear of the four refresh slots — the hand patch's choice of :52 is measured precedent.
initial_startonadd_compression_policy(which setsfixed_schedule = true), anchored forward the way the refresh policies anchor.- Extend the converge-on-startup sweep to compression jobs: select jobs whose schedule differs from the desired phase/fixed-schedule state,
alter_jobeach with the same per-job failure isolation. - Same identity rule as Narrow the hourly refresh window and phase the refresh family across the hour (#3012) #3024: key on the hypertable, never a job id.
Open design question to settle in the PR: whether all compression jobs share one slot (they compress different hypertables, so they do not contend on locks with each other — but ~70 simultaneous compression starts is an IO consideration), or whether they phase across a second grid offset from the refresh grid.
Interaction to preserve
ConvergeCompressionScheduleAsync already exists (#1778) and converges schedule_interval only; the new phase converge must compose with it rather than fight it — one function owning both properties is likely the cleaner end state.
Follow-on from #3012 / PR #3024; the live evidence base is recorded in that issue's comment trail.
Activity
Pin the compression policies to a phase grid clear of the refresh slo…
erikdarlingdata commentedon Sep 5, 2026
Claude posting for Erik Darling
Closed by #3041, merged to
devatb8797dcdb4c13cce669db062a3a9dca292426a4a. Closing by hand because a closing keyword does not fire againstdev.Compression policies now sit on a fixed phase grid —
:22-:29,:37-:44,:52-:59, the second half of each refresh slot with the heaviest refresh's slot excluded whole — instead of keeping TimescaleDB's finish-to-start scheduling and drifting by their own runtime every cycle.Two corrections to this issue's own framing, recorded because the fix is shaped by them.
The "convoy at alignment" model this was filed on does not hold at the current operating point. With the refresh window narrowed to 1 day, the window a refresh re-materializes and the chunks a compression policy finds eligible barely intersect, so alignment on the clock no longer means contention for the same data. A live collision was watched and was harmless for that structural reason, not by luck.
What remains is real and smaller: a boundary chunk exactly as wide as the running refresh's age, and the case originally measured in #3012, which never needed chunk overlap at all — a queued
AccessExclusiveLockbehind a heldAccessShareLockblocks every later shared request, so collector writes pile up behind a lock nobody holds. Both scale with refresh runtime rather than with alignment.The grid is therefore sized against the 864-second ceiling rather than the 194-second low, and the operating envelope is an assertion (
HeaviestHourlyRefreshObservedCeilingSeconds < RefreshPhaseStepMinutes * 60) rather than a sentence in a comment — a conditional downgrade written only in prose reads as unconditional to whoever finds it next. What invalidates it is the heaviest hourly refresh approaching its 900-second slot, at which point the grid needs redesigning rather than renumbering.The single shared minute this could have used is rejected on measurement.
CompressAfterDaysandChunkIntervalDaysare both 1 and TimescaleDB aligns 1-day chunks to the epoch, so every hypertable's newest closed chunk becomes eligible at the same UTC midnight. One shared minute would concentrate 70 simultaneous chunk rewrites once a day, withtimescaledb.max_background_workerssized atHypertableCount + 2so they would all genuinely run — a burst this change would have introduced rather than removed.