[Backport 2.x] Fix broken backward compatibility from 2.7 for IndexSorted field indices #10075

opensearch-trigger-bot · 2023-09-15T16:34:35Z

Backport d34b352 from #10045.

…ces (#10045) * Fix broken backward comparibility from 2.7 for IndexSorted field indices Signed-off-by: gashutos <gashutos@amazon.com> * Adding CHANGELOG Signed-off-by: gashutos <gashutos@amazon.com> * Update server/src/main/java/org/opensearch/index/IndexSettings.java Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: Chaitanya Gohel <104654647+gashutos@users.noreply.github.com> * Removing unwanted logs Signed-off-by: gashutos <gashutos@amazon.com> * Removing unwanted logs Signed-off-by: gashutos <gashutos@amazon.com> * Adding index sort as part of mixed cluster to test this scenario Signed-off-by: gashutos <gashutos@amazon.com> * Removing optimization disable logic Signed-off-by: gashutos <gashutos@amazon.com> * Correcting some comments & version check to before( V_2_7_0) instead onOrBefire(V_2_6_1) since Signed-off-by: gashutos <gashutos@amazon.com> * Resolving spotless check error Signed-off-by: gashutos <gashutos@amazon.com> * Fixing broken UT - last minute checkin without copile Signed-off-by: gashutos <gashutos@amazon.com> * Improving code coverage to make codcov happy Signed-off-by: gashutos <gashutos@amazon.com> * Correcting typos and adding more tests Signed-off-by: gashutos <gashutos@amazon.com> * Removing unwanted imports Signed-off-by: gashutos <gashutos@amazon.com> --------- Signed-off-by: gashutos <gashutos@amazon.com> Signed-off-by: Chaitanya Gohel <104654647+gashutos@users.noreply.github.com> Co-authored-by: Andriy Redko <drreta@gmail.com> (cherry picked from commit d34b352) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

github-actions · 2023-09-15T17:18:22Z

Compatibility status:

Checks if related components are compatible with change 53f4239

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

github-actions · 2023-09-15T17:23:43Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/25706/
CommitID: 748be94
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

gashutos · 2023-09-15T17:36:28Z

Unrelated failures, @reta @andrross can we re-trigger gradlew check ?

github-actions · 2023-09-15T18:24:26Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/25710/
CommitID: 748be94
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-09-15T19:42:33Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/25716/
CommitID: 748be94
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

gashutos · 2023-09-15T19:45:04Z

This failure is related to my changes.
org.opensearch.backwards.IndexingIT.testUpdateSnapshotStatus
checking....

github-actions · 2023-09-15T20:05:21Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/25720/
CommitID: 748be94
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

reta · 2023-09-15T21:02:12Z

This failure is related to my changes. org.opensearch.backwards.IndexingIT.testUpdateSnapshotStatus checking....

@gashutos identified the issue with tests:

Caused by: org.opensearch.transport.RemoteTransportException: [v1.3.13-0][127.0.0.1:42995][internal:index/shard/recovery/prepare_translog]
»  Caused by: java.lang.IllegalArgumentException: cannot change previous indexSort=<sortednumeric: "sortfield"> missingValue=9223372036854775807 selector=MIN type=LONG (from segment=_0(8.10.1):c8:[indexSort=<sortednumeric: "sortfield"> missingValue=9223372036854775807 selector=MIN type=LONG]:[diagnostics={os=Linux, timestamp=1694806619341, java.version=11.0.20, java.vm.version=11.0.20+8, lucene.version=8.10.1, source=flush, os.arch=amd64, java.runtime.version=11.0.20+8, os.version=5.4.254-175.358.amzn2int.x86_64, java.vendor=Eclipse Adoptium}]:[attributes={Lucene87StoredFieldsFormat.mode=BEST_SPEED}] :id=23nyyfkvqif07m3kcpvp96m6m) to new indexSort=<sortednumeric: "sortfield"> missingValue=2147483647 selector=MIN type=INT

The fix is in progress

github-actions · 2023-09-15T21:10:51Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/25724/
CommitID: 748be94
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

reta · 2023-09-16T15:21:08Z

The fix is in progress

Cleared out

Signed-off-by: Gohel <gashutos@88665a2fc937.ant.amazon.com> Co-authored-by: Gohel <gashutos@88665a2fc937.ant.amazon.com> (cherry picked from commit d7aa6dd) Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

server/src/main/java/org/opensearch/index/IndexSortConfig.java

github-actions · 2023-09-16T15:30:35Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/25748/
CommitID: 53f4239
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-09-16T16:21:06Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/25749/
CommitID: 20b306d

codecov · 2023-09-16T16:21:41Z

Codecov Report

Merging #10075 (20b306d) into 2.x (4c9042f) will decrease coverage by 0.23%.
Report is 6 commits behind head on 2.x.
The diff coverage is 68.42%.

@@             Coverage Diff              @@
##                2.x   #10075      +/-   ##
============================================
- Coverage     70.89%   70.67%   -0.23%     
+ Complexity    58291    58136     -155     
============================================
  Files          4810     4810              
  Lines        275140   275156      +16     
  Branches      40440    40442       +2     
============================================
- Hits         195074   194460     -614     
- Misses        63389    64069     +680     
+ Partials      16677    16627      -50

Files Changed	Coverage Δ
.../java/org/opensearch/common/collect/Iterators.java	`75.00% <ø> (ø)`
...n/java/org/opensearch/compress/ZstdCompressor.java	`52.94% <ø> (-17.65%)`	⬇️
...opensearch/core/common/breaker/CircuitBreaker.java	`75.00% <ø> (ø)`
.../core/common/breaker/CircuitBreakingException.java	`89.28% <ø> (ø)`
...search/core/common/breaker/NoopCircuitBreaker.java	`61.53% <0.00%> (ø)`
...earch/core/common/logging/LoggerMessageFormat.java	`20.14% <ø> (ø)`
...search/core/common/transport/TransportAddress.java	`79.48% <ø> (ø)`
.../org/opensearch/core/common/unit/ByteSizeUnit.java	`80.00% <ø> (ø)`
...g/opensearch/core/common/util/CollectionUtils.java	`77.06% <ø> (ø)`
...src/main/java/org/opensearch/core/index/Index.java	`100.00% <ø> (ø)`
... and 11 more

... and 486 files with indirect coverage changes

reta approved these changes Sep 15, 2023

View reviewed changes

gashutos mentioned this pull request Sep 15, 2023

[Backport 2.10] Fix broken backward compatibility from 2.7 for IndexSorted field indi… #10076

Merged

andrross approved these changes Sep 15, 2023

View reviewed changes

gashutos mentioned this pull request Sep 15, 2023

Removing idex sort fiild from testUpdateSnapshotStatus #10080

Closed

gashutos mentioned this pull request Sep 16, 2023

Integ test fix -> Fix broken backward compatibility from 2.7 for IndexSorted field indices #10045 #10087

Merged

Finxing broken UT (#10087)

20b306d

Signed-off-by: Gohel <gashutos@88665a2fc937.ant.amazon.com> Co-authored-by: Gohel <gashutos@88665a2fc937.ant.amazon.com> (cherry picked from commit d7aa6dd) Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

reta force-pushed the backport/backport-10045-to-2.x branch from 53f4239 to 20b306d Compare September 16, 2023 15:27

reta reviewed Sep 16, 2023

View reviewed changes

server/src/main/java/org/opensearch/index/IndexSortConfig.java Show resolved Hide resolved

reta merged commit 0e0cf0c into 2.x Sep 18, 2023

github-actions bot deleted the backport/backport-10045-to-2.x branch September 18, 2023 13:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Backport 2.x] Fix broken backward compatibility from 2.7 for IndexSorted field indices #10075

[Backport 2.x] Fix broken backward compatibility from 2.7 for IndexSorted field indices #10075

opensearch-trigger-bot bot commented Sep 15, 2023

github-actions bot commented Sep 15, 2023 •

edited

Loading

github-actions bot commented Sep 15, 2023

gashutos commented Sep 15, 2023

github-actions bot commented Sep 15, 2023

github-actions bot commented Sep 15, 2023

gashutos commented Sep 15, 2023

github-actions bot commented Sep 15, 2023

reta commented Sep 15, 2023

github-actions bot commented Sep 15, 2023

reta commented Sep 16, 2023

github-actions bot commented Sep 16, 2023

github-actions bot commented Sep 16, 2023

codecov bot commented Sep 16, 2023

[Backport 2.x] Fix broken backward compatibility from 2.7 for IndexSorted field indices #10075

[Backport 2.x] Fix broken backward compatibility from 2.7 for IndexSorted field indices #10075

Conversation

opensearch-trigger-bot bot commented Sep 15, 2023

github-actions bot commented Sep 15, 2023 • edited Loading

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Sep 15, 2023

Gradle Check (Jenkins) Run Completed with:

gashutos commented Sep 15, 2023

github-actions bot commented Sep 15, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 15, 2023

Gradle Check (Jenkins) Run Completed with:

gashutos commented Sep 15, 2023

github-actions bot commented Sep 15, 2023

Gradle Check (Jenkins) Run Completed with:

reta commented Sep 15, 2023

github-actions bot commented Sep 15, 2023

Gradle Check (Jenkins) Run Completed with:

reta commented Sep 16, 2023

github-actions bot commented Sep 16, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 16, 2023

Gradle Check (Jenkins) Run Completed with:

codecov bot commented Sep 16, 2023

Codecov Report

github-actions bot commented Sep 15, 2023 •

edited

Loading