Pagination: Add Close Cursor API in v2. #1660
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Yury-Fridlyand
requested review from
pjfitzgibbons,
ps48,
kavithacm,
derek-ho,
joshuali925,
dai-chen,
YANG-DB,
rupal-bq,
mengweieric,
vamsi-amazon,
Swiddis,
penghuo,
seankao-az,
MaxKsyunz,
anirudha,
forestmvey,
acarbonetto and
GumpacG
as code owners
Codecov Report
@@ Coverage Diff @@
## feature/pagination/integ #1660 +/- ##
===========================================================
Coverage 97.22% 97.23%
- Complexity 4217 4238 +21
===========================================================
Files 381 385 +4
Lines 10596 10626 +30
Branches 727 726 -1
===========================================================
+ Hits 10302 10332 +30
Misses 287 287
Partials 7 7
Flags with carried forward coverage won't be shown. Click here to find out more.
|
acarbonetto
reviewed
May 26, 2023
core/src/main/java/org/opensearch/sql/ast/tree/CloseCursor.java
Outdated
Show resolved
Hide resolved
core/src/main/java/org/opensearch/sql/ast/tree/CloseCursor.java
Outdated
Show resolved
Hide resolved
core/src/main/java/org/opensearch/sql/executor/execution/NonQueryPlan.java
Outdated
Show resolved
Hide resolved
core/src/main/java/org/opensearch/sql/executor/execution/QueryPlanFactory.java
Outdated
Show resolved
Hide resolved
core/src/main/java/org/opensearch/sql/executor/execution/QueryPlanFactory.java
Outdated
Show resolved
Hide resolved
opensearch/src/main/java/org/opensearch/sql/opensearch/request/OpenSearchScrollRequest.java
Outdated
Show resolved
Hide resolved
opensearch/src/main/java/org/opensearch/sql/opensearch/request/OpenSearchScrollRequest.java
Outdated
Show resolved
Hide resolved
...ocol/src/main/java/org/opensearch/sql/protocol/response/format/NoQueryResponseFormatter.java
Outdated
Show resolved
Hide resolved
matthewryanwells
previously approved these changes
May 26, 2023
Yury-Fridlyand
commented
May 26, 2023
Yury-Fridlyand
commented
May 26, 2023
GumpacG
previously approved these changes
May 26, 2023
forestmvey
reviewed
May 26, 2023
legacy/src/main/java/org/opensearch/sql/legacy/plugin/RestSQLQueryAction.java
Outdated
Show resolved
Hide resolved
forestmvey
previously approved these changes
May 26, 2023
MaxKsyunz
reviewed
May 27, 2023
There was a problem hiding this comment.
Thank you @Yury-Fridlyand!
What happens when closing the cursor fails?
core/src/main/java/org/opensearch/sql/executor/execution/QueryPlanFactory.java
Outdated
Show resolved
Hide resolved
| public class CloseCursor extends UnresolvedPlan { | ||
|
|
||
| /** An instance of {@link Cursor} */ | ||
| private UnresolvedPlan child; |
There was a problem hiding this comment.
What is this child used for?
As far as I can see, this can be cursor id without loss of generality and would make the code more straight-forward.
There was a problem hiding this comment.
True.
I tried to follow this graph:
If I store here the cursor itself instead of a child plan, the planner logic should be a bit more complicated to create 2 nodes.
There was a problem hiding this comment.
I see how it works but it's hard to follow. I hope we can make it more explicit at some point later.
...ocol/src/main/java/org/opensearch/sql/protocol/response/format/NoQueryResponseFormatter.java
Outdated
Show resolved
Hide resolved
MitchellGale
previously approved these changes
May 29, 2023
Yury-Fridlyand
dismissed stale reviews from forestmvey, GumpacG, and matthewryanwells
via
96cf27b
Yury-Fridlyand
force-pushed
the
feature/pagination/close-cursor-api
branch
from
9100c62
to
96cf27b
Compare
Yury-Fridlyand
changed the base branch from
tmp-pagination-integ
to
tmp-pagination-integ2
Nothing. A exception is suppressed and not even logged. |
|
Seems that jacoco is failing |
|
Thanks, fixed. |
MaxKsyunz
previously approved these changes
May 29, 2023
acarbonetto
reviewed
May 29, 2023
core/src/main/java/org/opensearch/sql/executor/execution/CommandPlan.java
Show resolved
Hide resolved
acarbonetto
previously approved these changes
May 29, 2023
core/src/main/java/org/opensearch/sql/executor/execution/QueryPlanFactory.java
Outdated
Show resolved
Hide resolved
| return new ExecutionEngine.Schema(List.of()); | ||
| } | ||
|
|
||
| // TODO remove |
core/src/main/java/org/opensearch/sql/planner/physical/CursorCloseOperator.java
Outdated
Show resolved
Hide resolved
core/src/test/java/org/opensearch/sql/analysis/AnalyzerTest.java
Outdated
Show resolved
Hide resolved
Yury-Fridlyand
dismissed stale reviews from acarbonetto and MaxKsyunz
via
d7f44aa
acarbonetto
approved these changes
May 30, 2023
Yury-Fridlyand
changed the base branch from
tmp-pagination-integ2
to
feature/pagination/integ
Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>
Yury-Fridlyand
force-pushed
the
feature/pagination/close-cursor-api
branch
from
c8c4f17
to
92e9fb7
Compare
MaxKsyunz
approved these changes
May 30, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add close cursor API to V2 similar to V1. Refer to pagination doc.
This code branched of #1600. Once #1600 merged I'll update this branch and change base branch of this PR. Currently, it is a temporary branch to show the changes and approach.
Design
https://github.com/opensearch-project/sql/blob/9100c6285f033784b5a2e58f53d575ece360737f/docs/dev/Pagination-v2.md#close-cursor
Issues Resolved
New endpoint on
_plugins/_sql/close. Under the hood it works as a subsequent query request, reserialized the cursor, but instead of executing it, just closes to encodedscroll.TODOs
Return an error on repetitivecloseor on failureclosefailureCheck List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.