Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support download latest index metadata from remote #9477

Merged
merged 12 commits into from
Sep 2, 2023
Merged

Add support download latest index metadata from remote #9477

merged 12 commits into from
Sep 2, 2023

Conversation

linuxpi
Copy link
Collaborator

@linuxpi linuxpi commented Aug 22, 2023 •
edited
Loading

Description

Related Issues

Resolves #9758 #9341

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@opensearch-trigger-bot
Copy link
Contributor

Compatibility status:

Checks if related components are compatible with change dd75a22

Incompatible components

Incompatible components: [https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/security-analytics.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

shwetathareja
shwetathareja reviewed Aug 23, 2023
View reviewed changes
Copy link
Member

@shwetathareja shwetathareja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @linuxpi for the changes. Left few comments.

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3BlobContainer.java Outdated Show resolved Hide resolved
server/src/main/java/org/opensearch/cluster/store/RemoteClusterStateService.java Outdated Show resolved Hide resolved
server/src/main/java/org/opensearch/cluster/store/RemoteClusterStateService.java Outdated Show resolved Hide resolved
server/src/main/java/org/opensearch/cluster/store/RemoteClusterStateService.java Outdated Show resolved Hide resolved
server/src/main/java/org/opensearch/cluster/store/RemoteClusterStateService.java Outdated Show resolved Hide resolved
server/src/main/java/org/opensearch/cluster/store/RemoteClusterStateService.java Outdated Show resolved Hide resolved
@linuxpi linuxpi marked this pull request as ready for review August 30, 2023 16:02
@linuxpi linuxpi force-pushed the remote-cluster-state-download-support branch from 8791d9e to 1b07539 Compare August 31, 2023 05:07
@github-actions
Copy link
Contributor

Compatibility status:

Checks if related components are compatible with change 1b07539

Incompatible components

Skipped components

Compatible components

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Sep 2, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Sep 2, 2023

Gradle Check (Jenkins) Run Completed with:

@linuxpi
Copy link
Collaborator Author

linuxpi commented Sep 2, 2023 •
edited
Loading

Test Failures

org.opensearch.remotestore.RemoteIndexRecoveryIT.testRerouteRecovery -- already muted in main
org.opensearch.snapshots.DedicatedClusterSnapshotRestoreIT.testIndexDeletionDuringSnapshotCreationInQueue -- Other PR where this is failing #9667 (comment)
org.opensearch.index.shard.RemoteStoreRefreshListenerTests.testAfterCommit -- #8947

We are good to merge

@linuxpi
Add support download latest index metadata from remote
cf6c801 
Signed-off-by: bansvaru <bansvaru@amazon.com>
@linuxpi
fix doc string
81a354d 
Signed-off-by: bansvaru <bansvaru@amazon.com>
@linuxpi
add failure scenerio UTs
842e047 
Signed-off-by: bansvaru <bansvaru@amazon.com>
@linuxpi
fix spotless errors for plugins
2d7b47a 
Signed-off-by: bansvaru <bansvaru@amazon.com>
@linuxpi
address PR comments
deea086 
Signed-off-by: bansvaru <bansvaru@amazon.com>
@linuxpi
address PR comments
19523d4 
Signed-off-by: bansvaru <bansvaru@amazon.com>
@linuxpi
fix spotless error
3486949 
Signed-off-by: bansvaru <bansvaru@amazon.com>
@linuxpi
correct naming
0929a1a 
Signed-off-by: bansvaru <bansvaru@amazon.com>
@linuxpi
Fix: correct limit checks for S3Container refactoring
a334513 
Signed-off-by: bansvaru <bansvaru@amazon.com>
@linuxpi
add docstring
971a654 
Signed-off-by: bansvaru <bansvaru@amazon.com>
@linuxpi
remove double logging
04d991d 
Signed-off-by: bansvaru <bansvaru@amazon.com>
@linuxpi
fix spotless error
0d1fcfb 
Signed-off-by: bansvaru <bansvaru@amazon.com>
@linuxpi linuxpi force-pushed the remote-cluster-state-download-support branch from 7a47048 to 0d1fcfb Compare September 2, 2023 12:29
@github-actions
Copy link
Contributor

github-actions bot commented Sep 2, 2023

Compatibility status:

Checks if related components are compatible with change 0d1fcfb

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

@github-actions
Copy link
Contributor

github-actions bot commented Sep 2, 2023

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testCancelPrimaryAllocation

@sachinpkale sachinpkale merged commit 422dd3c into opensearch-project:main Sep 2, 2023
15 checks passed
@sachinpkale sachinpkale added the backport 2.x Backport to 2.x branch label Sep 2, 2023
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 2, 2023
@github-actions
Add support download latest index metadata from remote (#9477)
33fa774 
---------

Signed-off-by: bansvaru <bansvaru@amazon.com>
(cherry picked from commit 422dd3c)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Sep 2, 2023
Merged
@linuxpi linuxpi deleted the remote-cluster-state-download-support branch September 4, 2023 06:25
sachinpkale pushed a commit that referenced this pull request Sep 4, 2023
@opensearch-trigger-bot @github-actions
Add support download latest index metadata from remote (#9477) (#9713)
08ba15d 
---------


(cherry picked from commit 422dd3c)

Signed-off-by: bansvaru <bansvaru@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
kaushalmahi12 pushed a commit to kaushalmahi12/OpenSearch that referenced this pull request Sep 12, 2023
@linuxpi @kaushalmahi12
Add support download latest index metadata from remote (opensearch-pr…
054b918 
…oject#9477)

---------

Signed-off-by: bansvaru <bansvaru@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
brusic pushed a commit to brusic/OpenSearch that referenced this pull request Sep 25, 2023
@linuxpi @brusic
Add support download latest index metadata from remote (opensearch-pr…
f11f0a2 
…oject#9477)

---------

Signed-off-by: bansvaru <bansvaru@amazon.com>
Signed-off-by: Ivan Brusic <ivan.brusic@flocksafety.com>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
@linuxpi @shiv0408
Add support download latest index metadata from remote (opensearch-pr…
87906a1 
…oject#9477)

---------

Signed-off-by: bansvaru <bansvaru@amazon.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
@gbbafna gbbafna mentioned this pull request Aug 8, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shwetathareja shwetathareja shwetathareja approved these changes

@sachinpkale sachinpkale sachinpkale approved these changes

@reta reta Awaiting requested review from reta reta is a code owner

@anasalkouz anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

@andrross andrross Awaiting requested review from andrross andrross is a code owner

@Bukhtawar Bukhtawar Awaiting requested review from Bukhtawar Bukhtawar is a code owner

@CEHENKLE CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

@dblock dblock Awaiting requested review from dblock dblock is a code owner

@gbbafna gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

@setiah setiah Awaiting requested review from setiah

@kartg kartg Awaiting requested review from kartg

@kotwanikunal kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

@mch2 mch2 Awaiting requested review from mch2 mch2 is a code owner

@nknize nknize Awaiting requested review from nknize nknize is a code owner

@owaiskazi19 owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

@Rishikesh1159 Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

@ryanbogan ryanbogan Awaiting requested review from ryanbogan

@saratvemulapalli saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

@dreamer-89 dreamer-89 Awaiting requested review from dreamer-89

@tlfeng tlfeng Awaiting requested review from tlfeng

@VachaShah VachaShah Awaiting requested review from VachaShah VachaShah is a code owner

@dbwiddis dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

@sohami sohami Awaiting requested review from sohami sohami is a code owner

@msfroh msfroh Awaiting requested review from msfroh msfroh is a code owner

Assignees
No one assigned
Labels
backport 2.x Backport to 2.x branch skip-changelog
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Remote Cluster State] Add support to download latest IndexMetadata from remote for a given cluster UUID
3 participants
@linuxpi @sachinpkale @shwetathareja