Skip to content

Test concurrent cloning/slicing of IndexInputs #51032

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

DaveCTurner
Copy link
Contributor

We test the behaviour of an IndexInput using randomReadAndSlice() which
uses a wide varity of different access methods to read the data. Lucene
sometimes calls clone() and slice() concurrently, although it does ensure
that there are no concurrent readers while these are being called. Today we do
not verify that our IndexInputs behave correctly under this kind of
concurrent access.

This commit extracts randomReadAndSlice() into a separate test harness for
more general consumption and adds support for concurrent cloning and slicing.

We test the behaviour of an `IndexInput` using `randomReadAndSlice()` which
uses a wide varity of different access methods to read the data. Lucene
sometimes calls `clone()` and `slice()` concurrently, although it does ensure
that there are no concurrent readers while these are being called. Today we do
not verify that our `IndexInput`s behave correctly under this kind of
concurrent access.

This commit extracts `randomReadAndSlice()` into a separate test harness for
more general consumption and adds support for concurrent cloning and slicing.
@DaveCTurner DaveCTurner added >non-issue :Distributed Indexing/Store Issues around managing unopened Lucene indices. If it touches Store.java, this is a likely label. labels Jan 15, 2020
@DaveCTurner DaveCTurner requested review from tlrx and ywelsch January 15, 2020 11:10
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (:Distributed/Store)

@DaveCTurner
Copy link
Contributor Author

@elasticmachine please run elasticsearch-ci/1
@elasticmachine please run elasticsearch-ci/default-distro

Copy link
Member

@tlrx tlrx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, this is a nice addition to the tests, thanks David

@DaveCTurner DaveCTurner merged commit cd72774 into elastic:feature/searchable-snapshots Jan 16, 2020
@DaveCTurner DaveCTurner deleted the 2020-01-15-test-concurrent-cloning-of-IndexInputs branch January 16, 2020 09:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Indexing/Store Issues around managing unopened Lucene indices. If it touches Store.java, this is a likely label. >non-issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants