-
Notifications
You must be signed in to change notification settings - Fork 2.3k
[GRPC] Optimize source conversion in gRPC search hits using zero-copy BytesRef #19280
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
❌ Gradle check result for f87cce9: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
… BytesRef Signed-off-by: karenx <karenx@uber.com>
ce0370d to
b13b325
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #19280 +/- ##
============================================
- Coverage 72.88% 72.82% -0.07%
+ Complexity 69841 69798 -43
============================================
Files 5673 5673
Lines 320756 320756
Branches 46370 46370
============================================
- Hits 233796 233590 -206
- Misses 68102 68272 +170
- Partials 18858 18894 +36 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
...c/src/main/java/org/opensearch/transport/grpc/proto/response/search/SearchHitProtoUtils.java
Show resolved
Hide resolved
Signed-off-by: Karen X <karenxyr@gmail.com>
|
❌ Gradle check result for dfa762f: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 8b9e86a: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: karenx <karenx@uber.com>
8b9e86a to
caf9cab
Compare
|
❌ Gradle check result for caf9cab: null Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Karen X <karenxyr@gmail.com>
|
❌ Gradle check result for eec47c3: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Karen X <karenxyr@gmail.com>
… BytesRef (opensearch-project#19280) * [GRPC] Optimize source conversion in gRPC search hits using zero-copy BytesRef Signed-off-by: karenx <karenx@uber.com> * code cov Signed-off-by: karenx <karenx@uber.com> * resolve merge conflicts Signed-off-by: karenx <karenx@uber.com> --------- Signed-off-by: karenx <karenx@uber.com> Signed-off-by: Karen X <karenxyr@gmail.com> Co-authored-by: karenx <karenx@uber.com>
… BytesRef (opensearch-project#19280) * [GRPC] Optimize source conversion in gRPC search hits using zero-copy BytesRef Signed-off-by: karenx <karenx@uber.com> * code cov Signed-off-by: karenx <karenx@uber.com> * resolve merge conflicts Signed-off-by: karenx <karenx@uber.com> --------- Signed-off-by: karenx <karenx@uber.com> Signed-off-by: Karen X <karenxyr@gmail.com> Co-authored-by: karenx <karenx@uber.com> Signed-off-by: Ankit Jain <jainankitk@apache.org>
… BytesRef (opensearch-project#19280) * [GRPC] Optimize source conversion in gRPC search hits using zero-copy BytesRef Signed-off-by: karenx <karenx@uber.com> * code cov Signed-off-by: karenx <karenx@uber.com> * resolve merge conflicts Signed-off-by: karenx <karenx@uber.com> --------- Signed-off-by: karenx <karenx@uber.com> Signed-off-by: Karen X <karenxyr@gmail.com> Co-authored-by: karenx <karenx@uber.com> Signed-off-by: Ankit Jain <jainankitk@apache.org>
… BytesRef (opensearch-project#19280) * [GRPC] Optimize source conversion in gRPC search hits using zero-copy BytesRef Signed-off-by: karenx <karenx@uber.com> * code cov Signed-off-by: karenx <karenx@uber.com> * resolve merge conflicts Signed-off-by: karenx <karenx@uber.com> --------- Signed-off-by: karenx <karenx@uber.com> Signed-off-by: Karen X <karenxyr@gmail.com> Co-authored-by: karenx <karenx@uber.com>
… BytesRef (opensearch-project#19280) * [GRPC] Optimize source conversion in gRPC search hits using zero-copy BytesRef Signed-off-by: karenx <karenx@uber.com> * code cov Signed-off-by: karenx <karenx@uber.com> * resolve merge conflicts Signed-off-by: karenx <karenx@uber.com> --------- Signed-off-by: karenx <karenx@uber.com> Signed-off-by: Karen X <karenxyr@gmail.com> Co-authored-by: karenx <karenx@uber.com>
… BytesRef (opensearch-project#19280) * [GRPC] Optimize source conversion in gRPC search hits using zero-copy BytesRef Signed-off-by: karenx <karenx@uber.com> * code cov Signed-off-by: karenx <karenx@uber.com> * resolve merge conflicts Signed-off-by: karenx <karenx@uber.com> --------- Signed-off-by: karenx <karenx@uber.com> Signed-off-by: Karen X <karenxyr@gmail.com> Co-authored-by: karenx <karenx@uber.com>
Description
Current Behavior
The current implementation of gRPC search hit source conversion in
SearchHitProtoUtils.processSource()always usesByteString.copyFrom()to create Protocol Buffer byte strings, which always creates a defensive copy of the underlying byte array for every search hit, regardless of the source type. This results in unnecessary memory copying and allocation overhead.Proposed Enhancement
This PR implements a zero-copy optimization for
BytesArraysources, which:a) Detects when the source is a
BytesArrayinstance and usingUnsafeByteOperations.unsafeWrap()forBytesArraysourcesb) Maintains backward compatibility by falling back to
ByteString.copyFrom()for otherBytesReferenceimplementationsExpected benefits
This should improve:
BytesArraysourcesRelated Issues
Resolves #19310
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.