Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server: prefetch point-get keys for multi-statement queries. #18155

Merged
merged 7 commits into from
Jun 23, 2020

Conversation

coocood
Copy link
Member

@coocood coocood commented Jun 22, 2020

What problem does this PR solve?

Statements in a multi-statement query are executed one by one currently.
If there are many point plan statements, we can prefetch the point keys, cache them in the transaction snapshot cache. reduce RPC cost and latency.

What is changed and how it works?

Proposal: xxx

What's Changed:

  • Remove the QueryCtx interface.
  • Build PointGet plans for a multi-statement query before execution.
  • Extract point keys and call BatchGet.

How it works:

  • Later execution will get the key from the cache, avoid Get RPC call.

Check List

Tests

  • Unit test

Release note

  • prefetch point-get keys for multi-statement queries.

@coocood coocood requested review from a team as code owners June 22, 2020 03:16
@coocood coocood requested review from XuHuaiyu and removed request for a team June 22, 2020 03:16
@lysu lysu requested review from tiancaiamao, lysu, bobotu and jackysp June 22, 2020 03:18
@lysu lysu unassigned lysu, jackysp and bobotu Jun 22, 2020
@github-actions github-actions bot added sig/execution SIG execution sig/planner SIG: Planner labels Jun 22, 2020
Copy link
Contributor

@lysu lysu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

continue
}
// Only support Update for now.
// TODO: support other point plans.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could support select/delete. It won't be very complex.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to make the change as minimum as possible to make the review work easier.

server/conn.go Outdated
return nil, nil
}
vars := cc.ctx.GetSessionVars()
if vars.TxnCtx.IsPessimistic && vars.TxnCtx.Isolation == ast.ReadCommitted {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe it has a limited effect when IsPessimistic == true && vars.TxnCtx.Isolation == RR? because it's still need lock per stmt(and lock request also can prefetch cache), but it's great for multi-stmt in side optimistic txn :D

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I should make it skip pessimistic transactions.

query := "update prefetch set c = c + 1 where a = 1 and b = 1;" +
"update prefetch set c = c + 1 where a = 2 and b = 2;" +
"update prefetch set c = c + 1 where a = 3 and b = 3;"
err = cc.handleQuery(ctx, query)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will you add a union scan case?

@coocood
Copy link
Member Author

coocood commented Jun 22, 2020

@lysu @jackysp PTAL

Copy link
Contributor

@lysu lysu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lysu lysu added the status/LGT1 Indicates that a PR has LGTM 1. label Jun 22, 2020
@codecov
Copy link

codecov bot commented Jun 22, 2020

Codecov Report

Merging #18155 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #18155   +/-   ##
===========================================
  Coverage   79.6358%   79.6358%           
===========================================
  Files           526        526           
  Lines        144425     144425           
===========================================
  Hits         115014     115014           
  Misses        20158      20158           
  Partials       9253       9253           

Copy link
Member

@jackysp jackysp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
PTAL @tiancaiamao

@tiancaiamao
Copy link
Contributor

LGTM

@tiancaiamao tiancaiamao added status/LGT3 The PR has already had 3 LGTM. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Jun 22, 2020
@jackysp
Copy link
Member

jackysp commented Jun 23, 2020

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Jun 23, 2020
@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@coocood merge failed.

@coocood
Copy link
Member Author

coocood commented Jun 23, 2020

/run-all-tests

@coocood coocood merged commit 9b16427 into pingcap:master Jun 23, 2020
@coocood coocood deleted the prefetch branch June 23, 2020 05:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/server sig/execution SIG execution sig/planner SIG: Planner status/can-merge Indicates a PR has been approved by a committer. status/LGT3 The PR has already had 3 LGTM. type/performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants