Skip to content

release: return processed grains from rollups - #194

Merged
NikolayS merged 2 commits into
NikolayS:mainfrom
samo-agent:agent/w4-191
Jul 27, 2026
Merged

release: return processed grains from rollups#194
NikolayS merged 2 commits into
NikolayS:mainfrom
samo-agent:agent/w4-191

Conversation

@NikolayS

@NikolayS NikolayS commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Fixes #191.

Unblocks #192.

Contract decision

ash.rollup_minute(integer) and ash.rollup_hour() now return the number of completed time grains processed, independent of how many (grain, datid) rows are upserted. This matches their catalog comments, the batch-limit model, and the operator-visible meaning of catch-up progress.

Each completed loop iteration counts once, including an empty grain: it consumes one batch slot and advances its watermark. A call with an already-caught-up watermark still returns exactly 0 because it processes no loop iterations.

Changes

  • increment rollup results once per processed minute/hour rather than by DML ROW_COUNT
  • add exact CI assertions for one and two databases, physical row counts, watermark deltas, empty grains, and already-caught-up rollup_minute(1000000)
  • align the historical rollup blueprint and 1.5-to-2.0 upgrade comment with the corrected contract
  • disclose the user-visible return correction in RELEASE_NOTES.md

Call-site audit

  • ash.rotate() stores the minute-rollup result but never reads or compares it. Exceptions and its independent (minute, datid) completeness query remain authoritative, so release: prevent cleanup from deadlocking rotation #185's truncation safety is unchanged.
  • ash.rollup_cleanup() does not consume either return value; it only shares the rollup advisory lock and continues returning physical deletion counts.
  • ash.status() does not call either rollup function; it reports physical table rows and watermarks directly.
  • pg_cron and documented external scheduler bodies execute bare SELECT statements and do not branch on the scalar.
  • existing tests either discard results, assert 0 on no-work/busy/incomplete paths, assert > 0, or assert 1 for a single grain/database. Those expectations remain valid. PR release: add behavioral feature-gate coverage #192's retained two-database assertion changes from RED to GREEN.

Validation

After the required hand rebase onto main 3af54a7:

The migration header was also corrected during rebase so it no longer broadly claims all admin/lifecycle functions are unchanged while documenting this intentional return-value correction.

@samo-agent samo-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified on exact rebased head 717545e. Issue #191 is reproducibly RED on main (one processed grain with two datids returns 2) and GREEN here (returns 1 while preserving two physical rows and 60/3600-second watermark movement). The exact added contract covers one/two datids, empty grains, and caught-up zero-work. The complete #185/#162 rotation regression passes. Independent static review found no blocker, and composing this fix with #192 makes all four feature-bearing surfaces pass on both PG17 and PG14 (8/8). Migration wording was corrected to describe the intentional return-value change without claiming all lifecycle behavior is unchanged. APPROVED.

@NikolayS
NikolayS marked this pull request as ready for review July 27, 2026 22:27
@NikolayS
NikolayS merged commit 2ccdf65 into NikolayS:main Jul 27, 2026
8 checks passed
@samo-agent
samo-agent deleted the agent/w4-191 branch July 27, 2026 22:32
samo-agent added a commit to samo-agent/pg_ash that referenced this pull request Jul 27, 2026
NikolayS#194 deliberately changed rollup_minute() from counting inserted database rows to counting every processed completed minute. The NikolayS#136 fixture starts twenty minutes in the past, so its unbounded setup call now correctly returns roughly twenty rather than the stale expected value 1. Limit that setup call to one grain and describe the assertion as one completed minute. The expected value remains 1 and every compacted-attribution assertion is unchanged. NikolayS#194 already records the return-contract change in RELEASE_NOTES.md.
NikolayS added a commit that referenced this pull request Jul 27, 2026
* test: expose compacted query attribution gaps

* fix: keep compacted query attribution honest

* docs: explain compacted query attribution

* test: bound query-attribution rollup setup

#194 deliberately changed rollup_minute() from counting inserted database rows to counting every processed completed minute. The #136 fixture starts twenty minutes in the past, so its unbounded setup call now correctly returns roughly twenty rather than the stale expected value 1. Limit that setup call to one grain and describe the assertion as one completed minute. The expected value remains 1 and every compacted-attribution assertion is unchanged. #194 already records the return-contract change in RELEASE_NOTES.md.

* test: verify rebased query-attribution interactions

* ci: trigger rebased pull-request verification

* ci: keep query-attribution gate runnable

GitHub Actions rejects workflow files above 512,000 bytes before creating a run. The #136 inline SQL raised test.yml from 506,109 to 535,600 bytes, which left #198 with no check suite. Move that exact 749-line payload to devel/tests/query_attribution_assert.sql and invoke it with psql -f; the SQL is byte-identical after removing YAML indentation and every assertion remains unchanged. The workflow is now 506,353 bytes.

---------

Co-authored-by: samo-agent <280144521+samo-agent@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rollup_minute() and rollup_hour() return per-database row counts, not processed grains

2 participants