-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add safeguard limits for file cache during node level allocation #8208
Add safeguard limits for file cache during node level allocation #8208
Conversation
840d282
to
98e9df7
Compare
Tagging previous reviewers: @andrross / @reta / @Bukhtawar |
Gradle Check (Jenkins) Run Completed with:
|
Codecov Report
@@ Coverage Diff @@
## main #8208 +/- ##
============================================
+ Coverage 70.85% 70.96% +0.10%
+ Complexity 56948 56942 -6
============================================
Files 4758 4759 +1
Lines 269382 269224 -158
Branches 39414 39405 -9
============================================
+ Hits 190877 191055 +178
+ Misses 62429 62002 -427
- Partials 16076 16167 +91
|
Gradle Check (Jenkins) Run Completed with:
|
server/src/main/java/org/opensearch/cluster/routing/RoutingNode.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/cluster/routing/RoutingPool.java
Outdated
Show resolved
Hide resolved
98e9df7
to
e190294
Compare
Gradle Check (Jenkins) Run Completed with:
|
@reta Resolved your comments. 3 green gradle checks in a row might be a sign :) |
@kotwanikunal my apologies, missed that somehow, will look first thing tomorrow morning |
...er/src/main/java/org/opensearch/cluster/routing/allocation/decider/DiskThresholdDecider.java
Show resolved
Hide resolved
...er/src/main/java/org/opensearch/cluster/routing/allocation/decider/DiskThresholdDecider.java
Outdated
Show resolved
Hide resolved
...er/src/main/java/org/opensearch/cluster/routing/allocation/decider/DiskThresholdDecider.java
Show resolved
Hide resolved
e190294
to
d94f6a9
Compare
Gradle Check (Jenkins) Run Completed with:
|
...er/src/main/java/org/opensearch/cluster/routing/allocation/decider/DiskThresholdDecider.java
Show resolved
Hide resolved
Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>
d94f6a9
to
f5ddcdc
Compare
Gradle Check (Jenkins) Run Completed with:
|
The backport to
To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-8208-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 91bfa01606974b947455fbc289e21a1aad096fa8
# Push it to GitHub
git push --set-upstream origin backport/backport-8208-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x Then, create a pull request where the |
…nsearch-project#8208) Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> (cherry picked from commit 91bfa01)
Will look into the backport after the followup PR. |
…nsearch-project#8208) Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>
…nsearch-project#8208) Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>
…nsearch-project#8208) Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>
…nsearch-project#8208) Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> Signed-off-by: sahil buddharaju <sahilbud@amazon.com>
…nsearch-project#8208) Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>
…nsearch-project#8208) Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> (cherry picked from commit 91bfa01)
* Add safeguard limits for file cache during node level allocation (#8208) Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> (cherry picked from commit 91bfa01) * Add restore level safeguards to prevent file cache oversubscription (#8606) Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> (cherry picked from commit a3aab67)
…nsearch-project#8208) Signed-off-by: Kunal Kotwani <kkotwani@amazon.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Description
remote
shard size on nodes and verifies if the shard can be safely allocated to the said nodesize of shard + sum(remote shards on the node) < 5 * (node cache size)
Related Issues
Resolves #7033
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.