Fix snapshotting searchable snapshot indexes

[andrross]

Two changes here: First, when snapshotting a searchable snapshot index (or snapshotting a full cluster that contains searchable snapshot indices) then we will snapshot the index metadata which includes the pointer to the original snapshot, but skip copying the data since it already exists in the source snapshot. Second, when restoring an index from a snapshot that is itself a searchable snapshot index, then it must be handled as such and restored as a searchable snapshot index.

Resolves #7204

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/14305/
• CommitID: 04b6e3b
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[Rishikesh1159]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/14305/
• CommitID: 04b6e3b
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

• What went wrong:
  Execution failed for task ':distribution:bwc:minor:buildBwcLinuxTar'.
  Building 2.7.0 didn't generate expected file /var/jenkins/workspace/gradle-check/search/distribution/bwc/minor/build/bwc/checkout-2.x/distribution/archives/linux-tar/build/distributions/opensearch-min-2.7.0-SNAPSHOT-linux-x64.tar.gz

@andrross could you please rebase the branch.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/14314/
• CommitID: cf82891
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/14316/
• CommitID: f05e8d2

[codecov-commenter]

Codecov Report

Merging #7247 (f05e8d2) into main (9f6c067) will decrease coverage by 0.59%.
The diff coverage is 47.05%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@             Coverage Diff              @@
##               main    #7247      +/-   ##
============================================
- Coverage     71.18%   70.59%   -0.59%     
+ Complexity    59887    59457     -430     
============================================
  Files          4859     4859              
  Lines        285505   285527      +22     
  Branches      41143    41150       +7     
============================================
- Hits         203223   201562    -1661     
- Misses        65924    67365    +1441     
- Partials      16358    16600     +242     

Impacted Files	Coverage Δ
...in/java/org/opensearch/index/shard/IndexShard.java	69.81% <0.00%> (-0.05%)	⬇️
...rg/opensearch/snapshots/SnapshotShardsService.java	63.27% <20.83%> (-3.70%)	⬇️
.../java/org/opensearch/snapshots/RestoreService.java	54.69% <66.66%> (-1.60%)	⬇️
...arch/repositories/blobstore/BlobStoreTestUtil.java	76.25% <73.91%> (+0.03%)	⬆️

... and 492 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-7247-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 e1da84d9d2e648066f3948906f935ce06495aac4
# Push it to GitHub
git push --set-upstream origin backport/backport-7247-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-7247-to-2.x.

[opensearch-trigger-bot]

The backport to 2.7 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.7 2.7
# Navigate to the new working tree
pushd ../.worktrees/backport-2.7
# Create a new branch
git switch --create backport/backport-7247-to-2.7
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 e1da84d9d2e648066f3948906f935ce06495aac4
# Push it to GitHub
git push --set-upstream origin backport/backport-7247-to-2.7
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.7

Then, create a pull request where the base branch is 2.7 and the compare/head branch is backport/backport-7247-to-2.7.

[Rishikesh1159]

@andrross looks like we need manual backport PR's for 2.x and 2.7