Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
244 changes: 233 additions & 11 deletions Darling/Darling.Tests/BaselineSupplyTests.cs

Large diffs are not rendered by default.

173 changes: 130 additions & 43 deletions Darling/Darling.Tests/DarlingAnomalyBaselineTests.cs

Large diffs are not rendered by default.

22 changes: 18 additions & 4 deletions Darling/Darling.Tests/RetentionTierRouterTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -202,10 +202,24 @@ public void DailySummary_RollupTier_RoutesOnlyTheQueryCte(RetentionTier tier, st
Assert.Contains($"FROM collect.{expectedRelation}", sql, StringComparison.Ordinal);
Assert.Contains("SELECT date_trunc('day', bucket) AS d, COUNT(DISTINCT query_hash) AS c", sql, StringComparison.Ordinal);

/* The routed CTE no longer reads the raw passthrough view... */
Assert.DoesNotContain("FROM v_query_stats", sql, StringComparison.Ordinal);

/* ...but the untouched sources still do. */
/* The routed CTE reads the raw passthrough view ONLY for the days past the rollup's last materialized
DAY (#3653: the daily rollup lags a day or two behind the clock, and those days used to print
unique_queries = 0 beside fresh raw numbers). Exactly one raw read, inside the UNION ALL tail, gated
on the ceiling; the rollup half stays the routed read for everything the rollup has. */
var rawReads = sql.Split("FROM v_query_stats").Length - 1;
Assert.Equal(1, rawReads);
var ceilingAt = sql.IndexOf("SELECT date_trunc('day', max(bucket)) AS last_day", StringComparison.Ordinal);
var rollupReadAt = sql.IndexOf("SELECT date_trunc('day', bucket) AS d, COUNT(DISTINCT query_hash) AS c", StringComparison.Ordinal);
var unionAt = sql.IndexOf("UNION ALL", StringComparison.Ordinal);
var rawReadAt = sql.IndexOf("FROM v_query_stats", StringComparison.Ordinal);
var tailGateAt = sql.IndexOf("AND collection_time >= COALESCE((SELECT last_day + INTERVAL '1 day' FROM queries_ceiling), $2)", StringComparison.Ordinal);
Assert.True(ceilingAt >= 0, "the rollup's last materialized day must be read per server");
Assert.Contains($"FROM collect.{expectedRelation}", sql[ceilingAt..rollupReadAt], StringComparison.Ordinal);
Assert.True(rollupReadAt > ceilingAt, "the rollup half follows the ceiling CTE");
Assert.True(unionAt > rollupReadAt && rawReadAt > unionAt, "the raw half is the UNION ALL tail, after the rollup half");
Assert.True(tailGateAt > rawReadAt, "the raw half must be gated to the days past the ceiling, or a day gets two rows");

/* The untouched sources still read raw. */
foreach (var untouched in new[] { "FROM v_wait_stats", "FROM v_deadlocks", "FROM v_cpu_utilization_stats", "FROM v_collection_log", "FROM v_memory_pressure_events" })
{
Assert.Contains(untouched, sql, StringComparison.Ordinal);
Expand Down
234 changes: 234 additions & 0 deletions Darling/Darling.Tests/RetiredBaselineAggregateTests.cs

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions Darling/Darling.Tests/TimescaleAggregateCompressionTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -415,7 +415,7 @@ static TimescaleSupport.AggregateCompressionState State(string view, long bytes,
State(TimescaleSupport.QueryStoreStatsIntervalDailyView, 33 * gib, 0),
/* Empty on this store (no writable Query Store primary): night zero. */
State(TimescaleSupport.QueryStoreStatsDailyView, 0, 0),
State(TimescaleSupport.PerfmonBaselineView, 0, 0),
State(TimescaleSupport.PerfmonIntervalBaselineView, 0, 0),
});

Assert.Equal(
Expand All @@ -429,7 +429,7 @@ static TimescaleSupport.AggregateCompressionState State(string view, long bytes,
ahead of the interval daily (11) ahead of the baseline (13). */
(TimescaleSupport.QueryStoreStatsDailyView, 0),
(TimescaleSupport.QueryStoreStatsIntervalDailyView, 0),
(TimescaleSupport.PerfmonBaselineView, 0),
(TimescaleSupport.PerfmonIntervalBaselineView, 0),
},
staged.ToArray());

Expand Down
4 changes: 2 additions & 2 deletions Darling/Darling.Tests/TimescaleSupportTests.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3471,8 +3471,8 @@ public void TheRefreshGridIsUnchanged_AndTheCompressionGridsOneInputFromItIsPinn
(TimescaleSupport.QueryStatsDbHourlyView, 2),
(TimescaleSupport.QueryStoreStatsIntervalHourlyView, 15),
(TimescaleSupport.QueryStoreStatsCorrectedHourlyView, 8),
(TimescaleSupport.PerfmonBaselineView, 3),
(TimescaleSupport.WaitStatsBaselineView, 5),
(TimescaleSupport.PerfmonIntervalBaselineView, 3),
(TimescaleSupport.WaitStatsIntervalBaselineView, 5),
(TimescaleSupport.SessionStatsBaselineView, 6),
(TimescaleSupport.QueryStatsBaselineView, 7),
(TimescaleSupport.BlockedProcessBaselineView, 9),
Expand Down
323 changes: 289 additions & 34 deletions Darling/PerformanceMonitor.Darling.Analysis/PgBaselineProvider.cs

Large diffs are not rendered by default.

34 changes: 34 additions & 0 deletions Darling/PerformanceMonitor.Darling.Storage/DailySummarySql.cs
Original file line number Diff line number Diff line change
Expand Up @@ -204,13 +204,47 @@ GROUP BY 1
/// one of its GROUP BY columns — the rollup preserves every distinct hash per bucket, so counting them per day
/// gives the identical answer raw would. The time column becomes <c>bucket</c>; parameter positions are
/// unchanged, so the caller binds the same three values either way.
///
/// <para><b>The days the rollup has not reached yet come from raw (#3653 A6, the <c>unique_queries = 0</c>
/// clause).</b> Both query rollups are materialized-only continuous aggregates (no
/// <c>materialized_only = false</c> on their CREATE, and TimescaleDB 2.13+ defaults real-time OFF), and
/// their refresh policies run behind the clock — the hourly's <c>end_offset</c> is an hour, the daily's a
/// day with a daily cadence, so the daily's newest complete bucket is one to two days old. A calendar whose
/// window starts past the hourly route's ceiling is routed to the DAILY rollup for the WHOLE window,
/// yesterday and today included, and there the LEFT JOIN found no row for those days: the outer
/// <c>COALESCE(q.c, 0)</c> then printed <c>unique_queries = 0</c> beside that same day's fresh wait, CPU
/// and deadlock numbers, which read the 30-day raw tables. Zero is not what was measured; "not
/// materialized yet" is. The CTE now takes the rollup for every day up to and including the rollup's last
/// materialized DAY for this server and raw <c>query_stats</c> for the days after it — raw keeps four days
/// and the lag is at most two, so those days are answered exactly, from the rows the rollup will
/// materialize tomorrow. The two halves are partitioned on the ceiling DAY, never on the hour, so no day
/// has two rows in <c>queries</c> (a <c>COUNT(DISTINCT)</c> cannot be summed across halves) and the
/// hourly tier's ceiling day reads its own partial rollup rather than a mix. Bounded by shape: the ceiling
/// is one indexed <c>max(bucket)</c> per server and the raw half is at most two days of one server.</para>
///
/// <para><b>What this does NOT do.</b> A day the rollup skipped BELOW its ceiling (the un-materialized
/// tail a &gt;24 h outage leaves, see <c>RetentionTierRouter</c>) still has no row and still prints 0; the
/// honest answer there is NULL ("not carried at this tier", contract rule 5), and that is a payload-shape
/// change — the reader, the model, the MCP tool and both calendars — which belongs to the census lane's
/// vocabulary rather than to a Storage-only edit. Lite's twin has no rollup tier and is untouched.</para>
/// </summary>
private static string QueriesCteForCagg(string relation) => $"""
queries_ceiling AS (
SELECT date_trunc('day', max(bucket)) AS last_day
FROM collect.{relation}
WHERE server_id = $1
),
queries AS (
SELECT date_trunc('day', bucket) AS d, COUNT(DISTINCT query_hash) AS c
FROM collect.{relation}
WHERE server_id = $1 AND bucket >= $2 AND bucket < $3
GROUP BY 1
UNION ALL
SELECT date_trunc('day', collection_time) AS d, COUNT(DISTINCT query_hash) AS c
FROM v_query_stats
WHERE server_id = $1 AND collection_time >= $2 AND collection_time < $3
AND collection_time >= COALESCE((SELECT last_day + INTERVAL '1 day' FROM queries_ceiling), $2)
GROUP BY 1
),
""";

Expand Down
18 changes: 18 additions & 0 deletions Darling/PerformanceMonitor.Darling.Storage/RetentionTierRouter.cs
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,24 @@ public static RetentionTier Resolve(DateTime nowUtc, DateTime windowStartUtc, bo
/// <c>min(bucket)</c> goes on reporting the original deep floor. The window then routes to the tier and is
/// served as complete with the gap silently inside it.</para>
///
/// <para><i>How big that hole is, read against TimescaleDB 2.28.1's mechanics (#3653 A6, the honest
/// read).</i> It is NOT the outage's length. The outage itself has no raw rows, so the rollup is correctly
/// empty there. What goes missing is the raw collected BETWEEN the last pre-outage refresh's window end
/// (<c>end_offset</c> behind the clock) and the moment collection stopped — at the hourly cadence at most
/// <c>end_offset</c> plus one <see cref="TimescaleSupport.HourlyRefreshScheduleInterval"/> of collections,
/// about two hours — because the first refresh after resume opens at <c>now - start_offset</c>, the
/// watermark jumps past that tail, and a real-time aggregate serves nothing below its watermark that was
/// never materialized (the hard partition <see cref="TimescaleSupport.BackfillBaselineAggregatesAsync"/>
/// documents). Any outage longer than <c>start_offset</c> less that tail does it. Nothing in the product
/// repairs it: the startup backfill and <c>--backfill-rollups</c> both measure the FLOOR, and a floor cannot
/// see a hole above itself; once raw retention passes the tail (four days on the rollup tier) the hole is
/// permanent. The repair that fits without a rung is a startup pass that finds raw-time gaps longer than
/// <see cref="TimescaleSupport.HourlyRefreshStartOffset"/> and runs <c>refresh_continuous_aggregate</c>
/// (forced) over <c>[gap start - the tail, gap end + start_offset]</c> for every hourly aggregate; it is
/// its own lane, because the gap scan has to be bounded by shape on the largest raw tables and the store
/// carries no collection-run ledger to read it from cheaply. Recorded here so the next reader does not
/// re-derive "a day of history" from the constant's name.</para>
///
/// <para><i>A window STRADDLING the floor is served partially, with no signal to the caller.</i> When the
/// window starts below the floor but no lower tier reaches further back, this returns the tier — which is
/// the correct CHOICE (raw would return less on a healthy store), but the part of the window below the
Expand Down
Loading
Loading