Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unreferenced file cleanup count to merge stats #10204

Merged
merged 1 commit into from
Oct 3, 2023

Conversation

RS146BIJAY
Copy link
Contributor

@RS146BIJAY RS146BIJAY commented Sep 25, 2023

Description

Adding stats for unreferenced files cleanup count on segment merge failure due to disk full.

unreferenced_file_cleanup.count: Number of times unreferenced files are cleaned up on segment merge failures.

"indices": {
        "merges": {
          "current": 0,
          "current_docs": 0,
          "current_size_in_bytes": 0,
          "total": 0,
          "total_time_in_millis": 0,
          "total_docs": 0,
          "total_size_in_bytes": 0,
          "total_stopped_time_in_millis": 0,
          "total_throttled_time_in_millis": 0,
          "total_auto_throttle_in_bytes": 120109614,
          "unreferenced_file_cleanup_count": 3
        }
      }

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 25, 2023

Compatibility status:

Checks if related components are compatible with change 6817c01

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git]

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@codecov
Copy link

codecov bot commented Sep 25, 2023

Codecov Report

Merging #10204 (6817c01) into main (d3bf230) will increase coverage by 0.03%.
Report is 4 commits behind head on main.
The diff coverage is 73.33%.

@@             Coverage Diff              @@
##               main   #10204      +/-   ##
============================================
+ Coverage     71.15%   71.18%   +0.03%     
- Complexity    58247    58306      +59     
============================================
  Files          4831     4831              
  Lines        274684   274704      +20     
  Branches      40026    40030       +4     
============================================
+ Hits         195443   195555     +112     
+ Misses        62884    62785      -99     
- Partials      16357    16364       +7     
Files Coverage Δ
...rch/extensions/rest/RestActionsRequestHandler.java 100.00% <100.00%> (ø)
...extensions/rest/RestInitializeExtensionAction.java 65.00% <100.00%> (-0.44%) ⬇️
...rch/extensions/rest/RestSendToExtensionAction.java 42.14% <100.00%> (ø)
.../main/java/org/opensearch/index/engine/Engine.java 74.65% <100.00%> (-1.59%) ⬇️
...in/java/org/opensearch/index/shard/IndexShard.java 70.17% <100.00%> (+0.13%) ⬆️
...va/org/opensearch/search/DefaultSearchContext.java 77.41% <0.00%> (-0.56%) ⬇️
...a/org/opensearch/extensions/ExtensionsManager.java 52.14% <62.50%> (+8.12%) ⬆️
...in/java/org/opensearch/index/merge/MergeStats.java 78.35% <70.00%> (-1.43%) ⬇️

... and 450 files with indirect coverage changes

@github-actions
Copy link
Contributor

github-actions bot commented Oct 1, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 1, 2023

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.snapshots.CloneSnapshotIT.testCloneShallowSnapshotIndex

@github-actions
Copy link
Contributor

github-actions bot commented Oct 3, 2023

Gradle Check (Jenkins) Run Completed with:

@RS146BIJAY RS146BIJAY force-pushed the merge-stats branch 2 times, most recently from 118af10 to 1cf7b41 Compare October 3, 2023 06:50
@github-actions
Copy link
Contributor

github-actions bot commented Oct 3, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 3, 2023

Gradle Check (Jenkins) Run Completed with:

@RS146BIJAY RS146BIJAY force-pushed the merge-stats branch 2 times, most recently from 56d5eb1 to dc9785d Compare October 3, 2023 07:38
@github-actions
Copy link
Contributor

github-actions bot commented Oct 3, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 3, 2023

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Rishav Sagar <rissag@amazon.com>
@github-actions
Copy link
Contributor

github-actions bot commented Oct 3, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 3, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 3, 2023

Gradle Check (Jenkins) Run Completed with:

@sachinpkale sachinpkale merged commit 2df9eed into opensearch-project:main Oct 3, 2023
RS146BIJAY added a commit to RS146BIJAY/OpenSearch that referenced this pull request Oct 3, 2023
…t#10204)

Signed-off-by: Rishav Sagar <rissag@amazon.com>
Co-authored-by: Rishav Sagar <rissag@amazon.com>
RS146BIJAY added a commit to RS146BIJAY/OpenSearch that referenced this pull request Oct 3, 2023
…t#10204)

Signed-off-by: Rishav Sagar <rissag@amazon.com>
Co-authored-by: Rishav Sagar <rissag@amazon.com>
@RS146BIJAY RS146BIJAY deleted the merge-stats branch October 3, 2023 14:45
sachinpkale pushed a commit that referenced this pull request Oct 3, 2023
Signed-off-by: Rishav Sagar <rissag@amazon.com>
Co-authored-by: Rishav Sagar <rissag@amazon.com>
rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request Oct 3, 2023
…t#10204)

Signed-off-by: Rishav Sagar <rissag@amazon.com>
Co-authored-by: Rishav Sagar <rissag@amazon.com>
deshsidd pushed a commit to deshsidd/OpenSearch that referenced this pull request Oct 9, 2023
…t#10204)

Signed-off-by: Rishav Sagar <rissag@amazon.com>
Co-authored-by: Rishav Sagar <rissag@amazon.com>
vikasvb90 pushed a commit to vikasvb90/OpenSearch that referenced this pull request Oct 10, 2023
…t#10204)

Signed-off-by: Rishav Sagar <rissag@amazon.com>
Co-authored-by: Rishav Sagar <rissag@amazon.com>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
…t#10204)

Signed-off-by: Rishav Sagar <rissag@amazon.com>
Co-authored-by: Rishav Sagar <rissag@amazon.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants