Summary
The rollup pipeline intentionally compacts query attribution, but the 2.0 ash.top('query_id') surface presents the retained subset as an exact total.
rollup_minute() drops query IDs below rollup_min_backend_seconds.
rollup_hour() keeps only the top 100 query IDs.
_grain_by(..., 'query_id') emits only retained IDs.
ash.top() computes its denominator from those retained keys.
This can make a small retained query appear to represent 100% of load while most backend time disappears.
Verified on origin/main at 5aac2e3 with Postgres 18.3 through the real rollup_minute() path.
Reproduction
One raw minute contains:
total backend appearances = 26
query 111 = 3
query 222 = 2
query 333 = 1
no query_id = 20
With the default minimum of 3, the produced query_counts is only {111,3}. After raw data is removed:
ash.aas:
avg_aas = 0.43
backend_seconds = 26
ash.top('query_id'):
key = 111
avg_aas = 0.05
backend_seconds = 3
pct = 100.00
The correct retained-query share is 3 / 26 = 11.54%, and 23 seconds require an explicit unattributed/not-preserved residual.
Expected
For rollup-backed query breakdowns:
- compute the exact per-grain residual
sum(wait_counts) - sum(query_counts);
- expose that residual through the NULL/unattributed bucket;
- use the full backend total as the percentage denominator; and
- document that named query IDs in rollups are compacted top attribution.
A specific query_id filter cannot distinguish true zero from compacted-away attribution. It should use raw data and raise outside raw retention, or otherwise expose an explicit unavailable state.
Regression requirement
Generate the fixture through rollup_minute(), not by hand. After removing raw data, assert:
- sum of
top('query_id').backend_seconds equals ash.aas().backend_seconds;
- query 111 is 3 seconds and 11.54%;
- the NULL residual is 23 seconds; and
- a compacted-away explicit query filter is not reported as an exact rollup zero.
Summary
The rollup pipeline intentionally compacts query attribution, but the 2.0
ash.top('query_id')surface presents the retained subset as an exact total.rollup_minute()drops query IDs belowrollup_min_backend_seconds.rollup_hour()keeps only the top 100 query IDs._grain_by(..., 'query_id')emits only retained IDs.ash.top()computes its denominator from those retained keys.This can make a small retained query appear to represent 100% of load while most backend time disappears.
Verified on
origin/mainat5aac2e3with Postgres 18.3 through the realrollup_minute()path.Reproduction
One raw minute contains:
With the default minimum of 3, the produced
query_countsis only{111,3}. After raw data is removed:The correct retained-query share is
3 / 26 = 11.54%, and 23 seconds require an explicit unattributed/not-preserved residual.Expected
For rollup-backed query breakdowns:
sum(wait_counts) - sum(query_counts);A specific
query_idfilter cannot distinguish true zero from compacted-away attribution. It should use raw data and raise outside raw retention, or otherwise expose an explicit unavailable state.Regression requirement
Generate the fixture through
rollup_minute(), not by hand. After removing raw data, assert:top('query_id').backend_secondsequalsash.aas().backend_seconds;