Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Backport 2.x] Change default batch size of bulk API to Integer.MAX_VALUE #14881

Merged
merged 2 commits into from
Jul 23, 2024
Merged

[Backport 2.x] Change default batch size of bulk API to Integer.MAX_VALUE #14881

merged 2 commits into from
Jul 23, 2024

Conversation

opensearch-trigger-bot[bot]
Copy link
Contributor

Backport 97f26cc from #14725.

@github-actions
Deprecate batch_size parameter on bulk API (#14725)
50d32ab 
By default the full _bulk payload will be passed to ingest processors as a
batch, with any sub batching logic to be implemented by each processor if
necessary.

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
(cherry picked from commit 97f26cc)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 50d32ab: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 50d32ab: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@dblock
Copy link
Member

dblock commented Jul 22, 2024 

2024-07-22T21:43:09.9282635Z /home/runner/work/OpenSearch/OpenSearch/server/src/test/java/org/opensearch/ingest/IngestServiceTests.java:1988: error: cannot find symbol
2024-07-22T21:43:09.9284554Z 
2024-07-22T21:43:09.9285829Z         final List<Integer> dropHandler = new ArrayList<>();
2024-07-22T21:43:09.9286997Z                                               ^
2024-07-22T21:43:09.9288157Z   symbol:   class ArrayList
2024-07-22T21:43:09.9289050Z   location: class IngestServiceTests
2024-07-22T21:43:09.9290046Z > Task :server:compileTestJava

@dblock dblock mentioned this pull request Jul 22, 2024
Merged
3 tasks
@dblock
Copy link
Member

dblock commented Jul 22, 2024

Doesn't look like a fluke, @chishui

@chishui
Copy link
Contributor

chishui commented Jul 23, 2024 •
edited
Loading

@dblock import java.util.ArrayList; is missing in 2.x but in main, but I don't have permission to update backport PR, is it possible that you can help add this line to the file or grant me permission to this PR?

@gaobinlong
Fix test failure
0a97145 
Signed-off-by: Gao Binlong <gbinlong@amazon.com>
@github-actions GitHub Actions
Copy link
Contributor

❌ Gradle check result for 0a97145: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions GitHub Actions
Copy link
Contributor

✅ Gradle check result for 0a97145: SUCCESS

@codecov Codecov
Copy link

codecov bot commented Jul 23, 2024

Codecov Report

Attention: Patch coverage is 75.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 71.56%. Comparing base (7e7e775) to head (0a97145).
Report is 8 commits behind head on 2.x.

Files Patch % Lines
...pensearch/rest/action/document/RestBulkAction.java 50.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files 
@@             Coverage Diff              @@
##                2.x   #14881      +/-   ##
============================================
- Coverage     71.59%   71.56%   -0.04%     
+ Complexity    62621    62537      -84     
============================================
  Files          5110     5110              
  Lines        293685   293651      -34     
  Branches      42801    42791      -10     
============================================
- Hits         210265   210151     -114     
- Misses        65801    65851      +50     
- Partials      17619    17649      +30

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@gaobinlong gaobinlong merged commit ed24a14 into 2.x Jul 23, 2024
32 checks passed
@gaobinlong gaobinlong deleted the backport/backport-14725-to-2.x branch July 23, 2024 06:43
kkewwei pushed a commit to kkewwei/OpenSearch that referenced this pull request Jul 24, 2024
@opensearch-trigger-bot @github-actions
@gaobinlong @kkewwei
[Backport 2.x] Change default batch size of bulk API to Integer.MAX_V…
8f67cbb 
…ALUE (opensearch-project#14881)

* Deprecate batch_size parameter on bulk API (opensearch-project#14725)

By default the full _bulk payload will be passed to ingest processors as a
batch, with any sub batching logic to be implemented by each processor if
necessary.

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
(cherry picked from commit 97f26cc)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Fix test failure

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: kkewwei <kkewwei@163.com>
This was referenced Sep 6, 2024
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrross andrross andrross approved these changes

@gaobinlong gaobinlong gaobinlong approved these changes

@anasalkouz anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

@ashking94 ashking94 Awaiting requested review from ashking94 ashking94 is a code owner

@Bukhtawar Bukhtawar Awaiting requested review from Bukhtawar Bukhtawar is a code owner

@CEHENKLE CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

@dblock dblock Awaiting requested review from dblock dblock is a code owner

@dbwiddis dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

@gbbafna gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

@kotwanikunal kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

@mch2 mch2 Awaiting requested review from mch2 mch2 is a code owner

@msfroh msfroh Awaiting requested review from msfroh msfroh is a code owner

@nknize nknize Awaiting requested review from nknize nknize is a code owner

@owaiskazi19 owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

@reta reta Awaiting requested review from reta reta is a code owner

@Rishikesh1159 Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

@sachinpkale sachinpkale Awaiting requested review from sachinpkale sachinpkale is a code owner

@saratvemulapalli saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

@shwetathareja shwetathareja Awaiting requested review from shwetathareja shwetathareja is a code owner

@sohami sohami Awaiting requested review from sohami sohami is a code owner

@VachaShah VachaShah Awaiting requested review from VachaShah VachaShah is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@dblock @chishui @andrross @gaobinlong