Skip to content

fix(util): reject LiveDocs backing sets with bits beyond maxDoc - #16768

Open
salvatorecampagna wants to merge 1 commit into
apache:mainfrom
salvatorecampagna:fix/livedocs-maxdoc-bounds
Open

salvatorecampagna wants to merge 1 commit into
apache:mainfrom
salvatorecampagna:fix/livedocs-maxdoc-bounds

Conversation

@salvatorecampagna

@salvatorecampagna salvatorecampagna commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

TL;DR

DenseLiveDocs and SparseLiveDocs take a backing bit set that may be longer than maxDoc, but count it at its full length. A bit actually set past maxDoc makes deletedCount wrong and makes two of the iterators return doc ids that don't exist. Both builders now reject that.

Summary

Both classes accept a backing set that only has to be at least maxDoc long while length() returns maxDoc, and only applyMask respects that. So a bit set at or beyond maxDoc counts as a document: deletedCount goes wrong, and DenseLiveDocs.liveDocsIterator and SparseLiveDocs.deletedDocsIterator emit doc ids that don't exist. build can't catch it, deriving the count from an unbounded cardinality that still lands in [0, maxDoc].

Padding is fine and stays supported, since with those bits clear the counts and iterators already agree with the deletions. So this rejects set bits at or beyond maxDoc rather than requiring exact length, which leaves #16593's applyMask support alone and means the iterators need no bound: BitSetIterator only emits set bits. maxDoc is checked first because nextSetBit indexes the backing set.

Nothing in Lucene hits this, Lucene90LiveDocsFormat always passes exactly maxDoc. Shipped in 10.5.0, so it applies to branch_10x.

Testing

./gradlew :lucene:core:test --tests "org.apache.lucene.util.TestLiveDocs"
./gradlew :lucene:core:test --tests "org.apache.lucene.util.TestFixedBitSet" --tests "org.apache.lucene.util.TestSparseFixedBitSet"
./gradlew :lucene:core:test --tests "org.apache.lucene.codecs.lucene90.TestLucene90LiveDocsFormat"
./gradlew :lucene:core:test --tests "org.apache.lucene.index.TestPendingDeletes" --tests "org.apache.lucene.index.TestPendingSoftDeletes"
./gradlew :lucene:core:test --tests "org.apache.lucene.index.TestSoftDeletesDirectoryReaderWrapper" --tests "org.apache.lucene.index.TestSoftDeletesRetentionMergePolicy"
./gradlew :lucene:core:test --tests "org.apache.lucene.index.TestIndexWriterDelete"

@salvatorecampagna salvatorecampagna changed the title fix(util): require LiveDocs backing sets to be exactly maxDoc bits fix(util): reject LiveDocs backing sets with bits beyond maxDoc Oct 2, 2026
DenseLiveDocs and SparseLiveDocs accept a backing bit set longer than
maxDoc, but only `applyMask` bounds on it. A bit set at or beyond maxDoc
counts as a document: `deletedCount` goes wrong and two of the iterators
emit doc ids that do not exist. Padding with those bits clear is correct
today, so reject the set bits rather than the padding.

Not reachable through `Lucene90LiveDocsFormat`, which passes exactly maxDoc.
@salvatorecampagna
salvatorecampagna force-pushed the fix/livedocs-maxdoc-bounds branch from d54ca26 to 3f5ad03 Compare October 2, 2026 21:25
@salvatorecampagna
salvatorecampagna marked this pull request as ready for review October 3, 2026 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant