[Backport 2.x] Async Batch shards changes for GatewayAllocator

[shiv0408]

Changes for create/update/delete batches for batch mode for async fetch for both primary & replica. It also added the node scope setting to enable/ disable batch mode.

Signed-off-by: Gaurav Chandani chngau@amazon.com
Signed-off-by: Shivansh Arora hishiv@amazon.com
Signed-off-by: Aman Khare amkhar@amazon.com
(cherry picked from commit a69cd08 from #8746)

[github-actions]

❌ Gradle check result for 0a8b6ff: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for bc1660f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[shwetathareja]

@shiv0408 we will need documentation PR for the new settings as well

[github-actions]

❌ Gradle check result for bc1660f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❕ Gradle check result for 0eca30f: UNSTABLE

• TEST FAILURES:

      1 org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=indices.get_field_mapping/20_missing_field/Return empty object if field doesn't exist, but index does}

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

Attention: Patch coverage is 57.01220% with 141 lines in your changes are missing coverage. Please review.

Project coverage is 71.23%. Comparing base (0dd892c) to head (0eca30f).
Report is 201 commits behind head on 2.x.

Files	Patch %	Lines
...pensearch/gateway/ShardsBatchGatewayAllocator.java	60.97%	98 Missing and 14 partials ⚠️
.../cluster/routing/allocation/AllocationService.java	33.33%	9 Missing and 1 partial ⚠️
...ore/TransportNodesListShardStoreMetadataBatch.java	0.00%	10 Missing ⚠️
...er/routing/allocation/ExistingShardsAllocator.java	12.50%	7 Missing ⚠️
...va/org/opensearch/cluster/routing/RoutingNode.java	0.00%	1 Missing ⚠️
.../opensearch/gateway/BaseGatewayShardAllocator.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##                2.x   #13431      +/-   ##
============================================
- Coverage     71.28%   71.23%   -0.05%     
- Complexity    60145    60891     +746     
============================================
  Files          4957     5021      +64     
  Lines        282799   286714    +3915     
  Branches      41409    41896     +487     
============================================
+ Hits         201591   204239    +2648     
- Misses        64189    65270    +1081     
- Partials      17019    17205     +186     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.