Fix DLQ writer writing empty list #2931
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Krishna Kondaka <krishkdk@dev-dsk-krishkdk-2c-bd29c437.us-west-2.amazon.com>
kkondaka
requested review from
chenqi0805,
engechas,
graytaylor0,
dinujoh,
cmanning09,
asifsmohammed,
dlvenable and
oeyh
as code owners
graytaylor0
approved these changes
Jun 26, 2023
asifsmohammed
approved these changes
Jun 26, 2023
JonahCalvo
approved these changes
Jun 26, 2023
graytaylor0
approved these changes
Jun 26, 2023
dinujoh
reviewed
Jun 26, 2023
| @@ -258,6 +258,10 @@ private void handleFailures(final AccumulatingBulkRequest<BulkOperationWrapper, | |||
|
|
|||
| private BulkOperationRequestResponse handleRetry(final AccumulatingBulkRequest request, final BulkResponse response, int retryCount) throws InterruptedException { | |||
| final AccumulatingBulkRequest<BulkOperationWrapper, BulkRequest> bulkRequestForRetry = createBulkRequestForRetry(request, response); | |||
| if (bulkRequestForRetry.getOperationsCount() == 0) { | |||
There was a problem hiding this comment.
Can you give some insight what is this getOperationsCount ?
There was a problem hiding this comment.
It is the number of operations in the bulk request. If this operation fails (for any reason), we will retry. But if the request has 0 operations, then there is no need to retry and no reason to write to DLQ when we fail to write.
opensearch-trigger-bot bot
pushed a commit
that referenced
this pull request
Jul 10, 2023
Signed-off-by: Krishna Kondaka <krishkdk@dev-dsk-krishkdk-2c-bd29c437.us-west-2.amazon.com> Co-authored-by: Krishna Kondaka <krishkdk@dev-dsk-krishkdk-2c-bd29c437.us-west-2.amazon.com> (cherry picked from commit 1dd8bd3)
kkondaka
added a commit
that referenced
this pull request
Jul 10, 2023
Signed-off-by: Krishna Kondaka <krishkdk@dev-dsk-krishkdk-2c-bd29c437.us-west-2.amazon.com> Co-authored-by: Krishna Kondaka <krishkdk@dev-dsk-krishkdk-2c-bd29c437.us-west-2.amazon.com> (cherry picked from commit 1dd8bd3) Co-authored-by: kkondaka <41027584+kkondaka@users.noreply.github.com>
MaGonzalMayedo
pushed a commit
to MaGonzalMayedo/data-prepper
that referenced
this pull request
Jul 25, 2023
Signed-off-by: Krishna Kondaka <krishkdk@dev-dsk-krishkdk-2c-bd29c437.us-west-2.amazon.com> Co-authored-by: Krishna Kondaka <krishkdk@dev-dsk-krishkdk-2c-bd29c437.us-west-2.amazon.com> Signed-off-by: Marcos Gonzalez Mayedo <alemayed@amazon.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fix DLQ writer writing empty list.
DQL writer is writing empty list like this sometimes. Upon investigation, found that bulkOperation is being tried even the operations count is zero. This is resulting in writing empty DLQ List when the bulk operation fails.
Issues Resolved
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.