Skip to content

Bound build-buffer growth to svs.max_build_memory - #71

Merged
matt-welch merged 1 commit into
mainfrom
fix-is187-build-memory-growth
Oct 9, 2026
Merged

matt-welch merged 1 commit into
mainfrom
fix-is187-build-memory-growth

Conversation

@matt-welch

Copy link
Copy Markdown
Contributor

Description

For a table that has never been ANALYZEd, pg_class.reltuples is -1, which causes SvsMemoryCheckEstimatedBuildSize's pre-scan check to return immediately without checking anything. The scan-time buffer, SvsVectorBufferAppend, then doubles its backing allocation on every overflow with no reference to svs.max_build_memory or any other bound. The only real enforcement, SvsMemoryReserveBuild, runs on a forecast computed only after table_index_build_scan has already returned, so for an unanalyzed table there is no memory check in effect anywhere during the scan itself. In practice this lets the build buffer grow well past the configured ceiling before any check has a chance to refuse the build.

This change adds SvsMemoryCheckGrowthSize(uint64 requestedBytes) in src/svs_memory.h/.c, which errors against the existing BuildMemoryCeilingBytes() accessor with no reltuples-based skip condition, and calls it from SvsVectorBufferAppend's doubling branch before each repalloc_huge. This closes the one point in the growth path that was unchecked, while leaving SvsMemoryCheckEstimatedBuildSize's pre-scan reltuples <= 0 early return untouched. BuildCallback grows a second buffer, tidBuffer, in lockstep right after SvsVectorBufferAppend returns; no separate check is needed there, since the new check's error now fires (and unwinds the transaction) before that growth is ever reached.

A new TAP test, test/t/55_build_memory_unanalyzed_unbounded.pl, builds a never-analyzed table large enough to force several doublings and asserts that the build is refused before the scan completes, by checking that the post-scan "buffered N vectors" NOTICE is never logged. It also asserts the refusal names svs.max_build_memory and is not a plain allocation-size error, and that the refused build leaves no index behind. Backend RSS is polled during the build and reported as a diagnostic only, not asserted on, since at this ceiling the backend's baseline memory pushes total RSS over the threshold regardless of whether the fix is present, so RSS alone does not reliably distinguish fixed from unfixed behavior.

Related Issues

Fixes the build-memory admission gate not bounding scan-time buffer growth for tables with no (or stale) reltuples.

Type of Change

  • Bug fix
  • New feature
  • Refactor / code cleanup
  • Documentation update
  • Test addition or update
  • Build / CI change

Pre-Merge Checklist

Build

  • make completes without errors or warnings
  • make install completes successfully

Tests

  • If this PR introduces no new behavior: existing regression tests (make installcheck) and TAP tests (test/t/) pass with no failures
  • If this PR introduces new behavior: test cases covering it were added to test/sql/ and/or test/t/
  • If this PR adds a standalone unit-test module under test/modules/: it builds and passes (make -C test/modules/<module> installcheck)

Documentation

  • Relevant docs under docs/ updated if architecture or usage changed

Testing Notes

Added test/t/55_build_memory_unanalyzed_unbounded.pl and verified it both proves the bug and confirms the fix. With the fix reverted (git apply -R on the three-file diff, rebuilt and reinstalled), the test failed exactly one of seven assertions, the one checking that the post-scan NOTICE is never reached, while the other six assertions describing the eventual refusal still passed. With the fix reapplied, rebuilt and reinstalled, all seven assertions passed.

Also ran test/t/32_build_memory_gate.pl and test/t/38_build_large_vector_buffer.pl, the two existing tests that exercise this same code path, with the fix in place: both passed with no regressions. Full targeted run across all three files: 74 of 74 tests passed.

An unanalyzed table has reltuples <= 0, which makes the pre-scan memory
estimate return immediately without checking anything. The scan-time
vector buffer then doubled its allocation on every overflow with no
ceiling check of its own, and the only real enforcement ran on a post-
scan forecast computed after the scan, and therefore after every
doubling, had already happened.

 - Add SvsMemoryCheckGrowthSize, which errors against the build memory
   ceiling with no reltuples-based skip condition
 - Call it from the vector buffer's doubling branch before each repalloc,
   so growth is checked at the point it happens rather than only after
   the scan returns
 - Leave the existing pre-scan estimate check unchanged
 - Add a TAP test covering an unanalyzed table large enough to force
   several doublings

Signed-off-by: Matt Welch <matt.welch@intel.com>
@matt-welch
matt-welch requested a review from a team October 8, 2026 23:16

@asonje asonje left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix is correct and thoroughly verified

@matt-welch
matt-welch merged commit 57394fe into main Oct 9, 2026
5 checks passed
@matt-welch
matt-welch deleted the fix-is187-build-memory-growth branch October 9, 2026 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants