Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

…ch into deletepitservice

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

…ch into deletepitservice

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

…ch into deletepitservice

* Create pit service layer changes

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

…rdAlreadyReplicating() (opensearch-project#3943)

* Fixing flaky test failure happening for testShardAlreadyReplicating()

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

…e in 'server' directory (opensearch-project#3647)

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Updating Joda

Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>

* Fixing Tests

Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>

* update SHAs

Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

…Replications() (opensearch-project#3963)

* Fixing flaky test failure happening for testShardAlreadyReplicating()

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

…project#3658)

* Adding Latest Recevied checkpoint, replay checkpoint logic along with tests

Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com>

…#3837)

* Make HybridDirectory MMAP Extensions Configurable

Make the HybridDirectory's list of mmap extensions configurable
via index settings instead of being hard-coded to a specfic set.
Set defaults to the list of currently hard-coded values for
backwards compatibility.

Signed-off-by: Matt Weber <matt@mattweber.org>

* Add javadoc to INDEX_STORE_HYBRID_MMAP_EXTENSIONS.

Signed-off-by: Matt Weber <matt@mattweber.org>

…ogy in 'client' directory (opensearch-project#3966)

* Deprecate public methods and variables that contain 'master' terminology in 'client' directory

Signed-off-by: Tianli Feng <ftianli@amazon.com>

* Change 3 String type usages with 'master' to 'cluster manager'

Signed-off-by: Tianli Feng <ftianli@amazon.com>

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

…and add ITs. (opensearch-project#3743)

* Add null check when computing max segment version.

With segment replication enabled it is possible Lucene does not set the SegmentInfos
min segment version, leaving the default value as null.

Signed-off-by: Marc Handalian <handalm@amazon.com>

* Update peer recovery to set the translogUUID of replicas to the UUID generated on the primary.

This change updates the UUID when the translog is created to the value stored in the passed segment userdata.
This is to ensure during failover scenarios that the replica can be promoted and not have a uuid mismatch with the value stored in user data.

Signed-off-by: Marc Handalian <handalm@amazon.com>

* Wire up Segment Replication under the feature flag.

This PR wires up segment replication and adds some initial integration tests.

Signed-off-by: Marc Handalian <handalm@amazon.com>

* Add test to ensure replicas use primary translog uuid with segrep.

Signed-off-by: Marc Handalian <handalm@amazon.com>

* Update SegmentReplicationIT to assert previous commit points are valid and SegmentInfos can be built.
Fix nitpicks in PR feedback.

Signed-off-by: Marc Handalian <handalm@amazon.com>

* Fix test with Assert.fail to include a message.

Signed-off-by: Marc Handalian <handalm@amazon.com>

…earch-project#3893)

* opensearch-project#3842 adds a new parameter to the highlighter, the max_analyzer_offset. When this parameter is provided the highlight stops in its value. This prevents the highlighter to go beyond the index maxAnalyzedOffset.

Signed-off-by: Hauck <joaoh14@gmail.com>

* Adds a test for the new parameter

Signed-off-by: Hauck <joaoh14@gmail.com>

* Fix the test add in the previous commit;

Signed-off-by: Hauck <joaoh14@gmail.com>

* This was checking against the wrong field

Signed-off-by: Hauck <joaoh14@gmail.com>

* Only runs the test for the correct version

Signed-off-by: Hauck <joaoh14@gmail.com>

* Skips the test in Elasticsearch as well;

Signed-off-by: Hauck <joaoh14@gmail.com>

* Remove elastic 3.0 to test

Signed-off-by: Hauck <joaoh14@gmail.com>

* Skips all versions

Signed-off-by: Hauck <joaoh14@gmail.com>

* Remove unnecessary fields as pointed by @reta

Signed-off-by: Hauck <joaoh14@gmail.com>

* Compute if fieldMaxAnalyzedIsNotValid in the constructor as suggest by @reta

Signed-off-by: Hauck <joaoh14@gmail.com>

* As discussed, it is better to throws different exceptions for when the fieldMaxAnalyzed is not valid and for when it is disabled;

Signed-off-by: Hauck <joaoh14@gmail.com>

* hint what to do to allow highlight of bigger documents
Signed-off-by: Hauck <joaoh14@gmail.com>

* Let the user define the new parameter globally for all fields highlighted

Signed-off-by: Hauck <joaoh14@gmail.com>

* Change the  fieldMaxAnalyzedOffset Integer in order to use null when it is absent in highlight. This allows the error messages to much more precise, showing invalid for all negative numbers;

Signed-off-by: Hauck <joaoh14@gmail.com>

* Update javadocs and implements the stream methods for the new fields;

Signed-off-by: Hauck <joaoh14@gmail.com>

* builder.field do not accept null, so check before calling the method is necessary

Signed-off-by: Hauck <joaoh14@gmail.com>

* Only send and read the new fields if the version supports it

Signed-off-by: Hauck <joaoh14@gmail.com>

* the previous commit was checking the wrong field

Signed-off-by: Hauck <joaoh14@gmail.com>

* Check for version 3.0.0 instead of current version

Signed-off-by: Hauck <joaoh14@gmail.com>

* Update server/src/main/java/org/apache/lucene/search/uhighlight/CustomUnifiedHighlighter.java

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Hauck <joaoh14@gmail.com>

* Execute the test after version 3.0.0

Signed-off-by: Hauck <joaoh14@gmail.com>

Co-authored-by: Andriy Redko <drreta@gmail.com>

…ogy in 'test/framework' directory (opensearch-project#3978)

* Deprecate methods with 'master' name mainly in class 'InternalTestCluster', 'NodeRoles', and 'AbstractSnapshotIntegTestCase'

* Deprecate variables contain 'master' in the name in class 'InternalTestCluster'

Signed-off-by: Tianli Feng <ftianli@amazon.com>

…roject#3796)

* Parallelize stale blobs deletion during snapshot delete

Signed-off-by: Piyush Daftary <pdaftary@amazon.com>

* Adding test which throws exception

Signed-off-by: Piyush Daftary <pdaftary@amazon.com>

* Adusting identation for spotlessJavaCheck

Signed-off-by: Piyush Daftary <pdaftary@amazon.com>

* Adding more description to MAX_SHARD_BLOB_DELETE_BATCH_SIZE

Signed-off-by: Piyush Daftary <pdaftary@amazon.com>

* Renaming max_shard_blob_delete_batch_size to max_snapshot_shard_blob_delete_batch_size

Signed-off-by: Piyush Daftary <pdaftary@amazon.com>

A set of guidelines to decide whether a feature should be included in OpenSearch.

Signed-off-by: Rabi Panda <adnapibar@gmail.com>

Uses a shell environment where `-o pipefail` is available.

Closes opensearch-project#4005

Signed-off-by: Cole White <cwhite@wikimedia.org>

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

… into deletepitservice

…ch into deletepitservice