Add MinDocsCondition to Rollover API #7243

shinbay-almaz · 2023-04-19T11:30:21Z

Description

Added MinDocsCondition to prevent rollover if the index has less than specified number of docs

Issues Resolved

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2023-04-19T11:57:45Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/14290/
CommitID: 7948be3
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-04-19T12:31:43Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/14295/
CommitID: fb095e9
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-04-19T12:39:23Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/14296/
CommitID: 097e02c

codecov-commenter · 2023-04-19T12:42:11Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.71%. Comparing base (d7aa6dd) to head (c369090).
Report is 863 commits behind head on main.

❗ Current head c369090 differs from pull request most recent head 156a367. Consider uploading reports for the commit 156a367 to get more accurate results

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #7243      +/-   ##
============================================
- Coverage     71.02%   70.71%   -0.32%     
+ Complexity    58078    56029    -2049     
============================================
  Files          4825     4670     -155     
  Lines        274032   265699    -8333     
  Branches      39930    39015     -915     
============================================
- Hits         194628   187876    -6752     
+ Misses        63067    61927    -1140     
+ Partials      16337    15896     -441

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Poojita-Raj · 2023-04-20T14:57:44Z

...nt/rest-high-level/src/main/java/org/opensearch/client/indices/rollover/RolloverRequest.java

@@ -131,6 +132,18 @@ public RolloverRequest addMaxIndexSizeCondition(ByteSizeValue size) {
        return this;
    }

+    /**


same description as addMaxIndexDocsCondition. Specify what makes them different.

Poojita-Raj · 2023-04-20T15:00:49Z

server/src/main/java/org/opensearch/action/admin/indices/rollover/MinDocsCondition.java

+import java.io.IOException;
+
+/**
+ * Condition for minimum index docs. Evaluates to <code>true</code>


Nit: same as above - mention how it's different from maxDocs.

Poojita-Raj · 2023-04-20T15:12:28Z

server/src/main/java/org/opensearch/action/admin/indices/rollover/TransportRolloverAction.java

+                boolean metMinDocsCondition = rolloverRequest.getConditions().containsKey(MinDocsCondition.NAME)
+                    ? conditionResults.get(rolloverRequest.getConditions().get(MinDocsCondition.NAME))
+                    : true;
+                if (conditionResults.size() == 0 || (metConditions.size() > 0 && metMinDocsCondition)) {


Just to confirm: our expected behavior is that when a minDocsCondition is set -
any max condition and minDocsCondition must be true in order to rollover?

I think so. If there is no enough number of docs, it should not rollover

Poojita-Raj · 2023-04-20T15:22:02Z

...src/test/java/org/opensearch/action/admin/indices/rollover/TransportRolloverActionTests.java

        for (Map.Entry<String, Boolean> entry : results.entrySet()) {
            if (entry.getKey().equals(maxAgeCondition.toString())) {
                assertThat(entry.getValue(), equalTo(true));
            } else if (entry.getKey().equals(maxDocsCondition.toString())) {
                assertThat(entry.getValue(), equalTo(false));
            } else if (entry.getKey().equals(maxSizeCondition.toString())) {
                assertThat(entry.getValue(), equalTo(false));
+            } else if (entry.getKey().equals(minDocsCondition.toString())) {


In a test, it makes more sense to have deterministic expected outcomes. We could set the minDocsCondition value to 10, in which case the expected value will always be false.

Poojita-Raj · 2023-04-20T15:27:22Z

Thanks @shinbay-almaz for contributing by opening this PR!
Since it is an API that's customer facing, we should make sure to document this change. You can open another issue in opensearch-project/documentation-website requesting this parameter be documented for the rollover API once this change is merged in.

opensearch-trigger-bot · 2023-06-29T15:22:49Z

This PR was closed because it has been stalled for 7 days with no activity.

kotwanikunal · 2023-09-14T18:42:56Z

Apologies. This PR was auto closed without reaching a resolution from the maintainers.
Re-opening to move it forward.
Thanks for your contributions to OpenSearch!

github-actions · 2023-09-14T18:45:30Z

Compatibility status:

Checks if related components are compatible with change 2ef35db

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]

github-actions · 2023-09-14T19:10:58Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/25631/
CommitID: 9b12976
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

shinbay-almaz · 2023-09-17T17:24:46Z

@kotwanikunal Hi 👋 Sorry that I left in the middle. I had work in this summer. I think now I can finally complete this PR

github-actions · 2023-09-17T18:05:46Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/25754/
CommitID: 8329048
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Signed-off-by: Almaz Shinbay <shinbay.almaz@gmail.com>

github-actions · 2023-09-17T18:58:15Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/25755/
CommitID: 2ef35db
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-09-17T19:28:45Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/25756/
CommitID: 156a367
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

ticheng-aws · 2024-01-06T00:30:59Z

Hi @shinbay-almaz, the PR is stalled. Is this being worked upon? Feel free to reach out to maintainers for further reviews.

opensearch-trigger-bot · 2024-03-01T15:20:33Z

This PR is stalled because it has been open for 30 days with no activity.

stephen-crawford · 2024-03-29T20:48:46Z

It seems like this contribution is stalled. I would recommend we close it for now. @opensearch-project/opensearch-core could some close this please? Thanks

opensearch-trigger-bot · 2024-05-05T15:20:39Z

This PR is stalled because it has been open for 30 days with no activity.

kotwanikunal · 2024-05-28T14:57:16Z

Closing this PR since there are no updates.

shinbay-almaz requested review from reta, anasalkouz, andrross, Bukhtawar, CEHENKLE, dblock, gbbafna, setiah, kartg, kotwanikunal, mch2, nknize, owaiskazi19, Rishikesh1159, ryanbogan, saratvemulapalli, shwetathareja, dreamer-89, tlfeng, VachaShah and xuezhou25 as code owners April 19, 2023 11:30

Poojita-Raj reviewed Apr 20, 2023

View reviewed changes

opensearch-trigger-bot bot closed this Jun 29, 2023

kotwanikunal reopened this Sep 14, 2023

kotwanikunal requested review from peternied, dbwiddis, sachinpkale, sohami and msfroh as code owners September 14, 2023 18:42

Merge branch 'main' into min_docs_condition and resolve conflicts

8329048

Fix imports in MinDocsCondition.java

156a367

Signed-off-by: Almaz Shinbay <shinbay.almaz@gmail.com>

shinbay-almaz force-pushed the min_docs_condition branch from 2ef35db to 156a367 Compare September 17, 2023 18:34

ticheng-aws added the enhancement Enhancement or improvement to existing feature or request label Jan 6, 2024

opensearch-trigger-bot bot removed the stalled Issues that have stalled label Jan 14, 2024

opensearch-trigger-bot bot added stalled Issues that have stalled and removed stalled Issues that have stalled labels Mar 1, 2024

opensearch-trigger-bot bot added stalled Issues that have stalled and removed stalled Issues that have stalled labels May 5, 2024

kotwanikunal closed this May 28, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add MinDocsCondition to Rollover API #7243

Add MinDocsCondition to Rollover API #7243

shinbay-almaz commented Apr 19, 2023 •

edited

Loading

github-actions bot commented Apr 19, 2023

github-actions bot commented Apr 19, 2023

github-actions bot commented Apr 19, 2023

codecov-commenter commented Apr 19, 2023 •

edited by codecov bot

Loading

Poojita-Raj Apr 20, 2023

Poojita-Raj Apr 20, 2023

Poojita-Raj Apr 20, 2023

shinbay-almaz Apr 22, 2023

Poojita-Raj Apr 20, 2023

Poojita-Raj commented Apr 20, 2023

opensearch-trigger-bot bot commented Jun 29, 2023

kotwanikunal commented Sep 14, 2023

github-actions bot commented Sep 14, 2023 •

edited

Loading

github-actions bot commented Sep 14, 2023

shinbay-almaz commented Sep 17, 2023

github-actions bot commented Sep 17, 2023

github-actions bot commented Sep 17, 2023

github-actions bot commented Sep 17, 2023

ticheng-aws commented Jan 6, 2024

opensearch-trigger-bot bot commented Mar 1, 2024

stephen-crawford commented Mar 29, 2024

opensearch-trigger-bot bot commented May 5, 2024

kotwanikunal commented May 28, 2024

Add MinDocsCondition to Rollover API #7243

Add MinDocsCondition to Rollover API #7243

Conversation

shinbay-almaz commented Apr 19, 2023 • edited Loading

Description

Issues Resolved

Check List

github-actions bot commented Apr 19, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Apr 19, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Apr 19, 2023

Gradle Check (Jenkins) Run Completed with:

codecov-commenter commented Apr 19, 2023 • edited by codecov bot Loading

Codecov Report

Poojita-Raj Apr 20, 2023

Choose a reason for hiding this comment

Poojita-Raj Apr 20, 2023

Choose a reason for hiding this comment

Poojita-Raj Apr 20, 2023

Choose a reason for hiding this comment

shinbay-almaz Apr 22, 2023

Choose a reason for hiding this comment

Poojita-Raj Apr 20, 2023

Choose a reason for hiding this comment

Poojita-Raj commented Apr 20, 2023

opensearch-trigger-bot bot commented Jun 29, 2023

kotwanikunal commented Sep 14, 2023

github-actions bot commented Sep 14, 2023 • edited Loading

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Sep 14, 2023

Gradle Check (Jenkins) Run Completed with:

shinbay-almaz commented Sep 17, 2023

github-actions bot commented Sep 17, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 17, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 17, 2023

Gradle Check (Jenkins) Run Completed with:

ticheng-aws commented Jan 6, 2024

opensearch-trigger-bot bot commented Mar 1, 2024

stephen-crawford commented Mar 29, 2024

opensearch-trigger-bot bot commented May 5, 2024

kotwanikunal commented May 28, 2024

shinbay-almaz commented Apr 19, 2023 •

edited

Loading

codecov-commenter commented Apr 19, 2023 •

edited by codecov bot

Loading

github-actions bot commented Sep 14, 2023 •

edited

Loading