Repository navigation
Bound build-buffer growth to svs.max_build_memory - #71
Merged
Merged
Conversation
An unanalyzed table has reltuples <= 0, which makes the pre-scan memory estimate return immediately without checking anything. The scan-time vector buffer then doubled its allocation on every overflow with no ceiling check of its own, and the only real enforcement ran on a post- scan forecast computed after the scan, and therefore after every doubling, had already happened. - Add SvsMemoryCheckGrowthSize, which errors against the build memory ceiling with no reltuples-based skip condition - Call it from the vector buffer's doubling branch before each repalloc, so growth is checked at the point it happens rather than only after the scan returns - Leave the existing pre-scan estimate check unchanged - Add a TAP test covering an unanalyzed table large enough to force several doublings Signed-off-by: Matt Welch <matt.welch@intel.com>
asonje
approved these changes
Oct 9, 2026
asonje
left a comment
Contributor
There was a problem hiding this comment.
Fix is correct and thoroughly verified
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
For a table that has never been
ANALYZEd,pg_class.reltuplesis-1, which causesSvsMemoryCheckEstimatedBuildSize's pre-scan check to return immediately without checking anything. The scan-time buffer,SvsVectorBufferAppend, then doubles its backing allocation on every overflow with no reference tosvs.max_build_memoryor any other bound. The only real enforcement,SvsMemoryReserveBuild, runs on a forecast computed only aftertable_index_build_scanhas already returned, so for an unanalyzed table there is no memory check in effect anywhere during the scan itself. In practice this lets the build buffer grow well past the configured ceiling before any check has a chance to refuse the build.This change adds
SvsMemoryCheckGrowthSize(uint64 requestedBytes)insrc/svs_memory.h/.c, which errors against the existingBuildMemoryCeilingBytes()accessor with noreltuples-based skip condition, and calls it fromSvsVectorBufferAppend's doubling branch before eachrepalloc_huge. This closes the one point in the growth path that was unchecked, while leavingSvsMemoryCheckEstimatedBuildSize's pre-scanreltuples <= 0early return untouched.BuildCallbackgrows a second buffer,tidBuffer, in lockstep right afterSvsVectorBufferAppendreturns; no separate check is needed there, since the new check's error now fires (and unwinds the transaction) before that growth is ever reached.A new TAP test,
test/t/55_build_memory_unanalyzed_unbounded.pl, builds a never-analyzed table large enough to force several doublings and asserts that the build is refused before the scan completes, by checking that the post-scan "buffered N vectors" NOTICE is never logged. It also asserts the refusal namessvs.max_build_memoryand is not a plain allocation-size error, and that the refused build leaves no index behind. Backend RSS is polled during the build and reported as a diagnostic only, not asserted on, since at this ceiling the backend's baseline memory pushes total RSS over the threshold regardless of whether the fix is present, so RSS alone does not reliably distinguish fixed from unfixed behavior.Related Issues
Fixes the build-memory admission gate not bounding scan-time buffer growth for tables with no (or stale)
reltuples.Type of Change
Pre-Merge Checklist
Build
makecompletes without errors or warningsmake installcompletes successfullyTests
make installcheck) and TAP tests (test/t/) pass with no failurestest/sql/and/ortest/t/test/modules/: it builds and passes (make -C test/modules/<module> installcheck)Documentation
docs/updated if architecture or usage changedTesting Notes
Added
test/t/55_build_memory_unanalyzed_unbounded.pland verified it both proves the bug and confirms the fix. With the fix reverted (git apply -Ron the three-file diff, rebuilt and reinstalled), the test failed exactly one of seven assertions, the one checking that the post-scan NOTICE is never reached, while the other six assertions describing the eventual refusal still passed. With the fix reapplied, rebuilt and reinstalled, all seven assertions passed.Also ran
test/t/32_build_memory_gate.plandtest/t/38_build_large_vector_buffer.pl, the two existing tests that exercise this same code path, with the fix in place: both passed with no regressions. Full targeted run across all three files: 74 of 74 tests passed.