Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executor: fix build table reader for index join on partition table #19210

Merged
merged 3 commits into from
Aug 20, 2020

Conversation

tiancaiamao
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #19145

Problem Summary:

In IndexLookUpMergeJoin, the inner join table reader executor is built at runtime.
The inner table may be a partition table, but the old logic dataReaderBuilder.buildTableReaderForIndexJoin does not handle that.

What is changed and how it works?

What's Changed:
How it Works:

  • Turn the executor TableReader into PartitionTable->TableReader in buildTableReaderForIndexJoin
  • Key range calculating logic is moved from dataReaderBuilder to Executor, because table ID is unknown at databReaderBuilder

This fix works on the master branch, 4.0 is fixed by #19151

Check List

Tests

  • Unit test

Release note

  • No release note

@tiancaiamao tiancaiamao requested a review from a team as a code owner August 14, 2020 09:34
@tiancaiamao tiancaiamao requested review from wshwsh12 and removed request for a team August 14, 2020 09:34
@tiancaiamao
Copy link
Contributor Author

PTAL @lzmhhh123 @lysu @imtbkcat

Copy link
Contributor

@lzmhhh123 lzmhhh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does IndexReader have the same problem?

@codecov
Copy link

codecov bot commented Aug 14, 2020

Codecov Report

Merging #19210 into master will decrease coverage by 0.0223%.
The diff coverage is 61.2903%.

@@               Coverage Diff                @@
##             master     #19210        +/-   ##
================================================
- Coverage   79.0998%   79.0774%   -0.0224%     
================================================
  Files           549        549                
  Lines        149678     149236       -442     
================================================
- Hits         118395     118012       -383     
+ Misses        21761      21690        -71     
- Partials       9522       9534        +12     

@tiancaiamao
Copy link
Contributor Author

tiancaiamao commented Aug 14, 2020

Does IndexReader have the same problem?

IndexReader have already handled the partition table as join inner table case.

But not all the cases are covered until now. As I've said on another PR #18862

In buildExecutorForIndexJoinInternal, there are some other cases, I'm not sure when the code run there:

switch v := plan.(type) {
case *plannercore.PhysicalTableReader:
case *plannercore.PhysicalIndexReader:
case *plannercore.PhysicalIndexLookUpReader:
case *plannercore.PhysicalUnionScan:
case *plannercore.PhysicalProjection:
case *plannercore.PhysicalSelection:
case *mockPhysicalIndexReader:
}

  • case *plannercore.PhysicalTableReader:
  • case *plannercore.PhysicalIndexReader:
  • case *plannercore.PhysicalIndexLookUpReader:
  • case *plannercore.PhysicalUnionScan:
  • case *plannercore.PhysicalProjection:
  • case *plannercore.PhysicalSelection:
  • case *mockPhysicalIndexReader:

I'll handle the remain cases later, after I solve this bug #19089

@lzmhhh123

Copy link
Contributor

@lzmhhh123 lzmhhh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 18, 2020
@lysu
Copy link
Contributor

lysu commented Aug 19, 2020

/approve

@ti-srebot ti-srebot removed the status/LGT1 Indicates that a PR has LGTM 1. label Aug 19, 2020
@ti-srebot ti-srebot added the status/LGT2 Indicates that a PR has LGTM 2. label Aug 19, 2020
@lysu
Copy link
Contributor

lysu commented Aug 19, 2020

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 19, 2020
@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@tiancaiamao merge failed.

@zz-jason
Copy link
Member

/merge

@ti-srebot
Copy link
Contributor

Your auto merge job has been accepted, waiting for:

  • 19266
  • 19248

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@tiancaiamao merge failed.

@tiancaiamao
Copy link
Contributor Author

/run-integration-copr-test

@tiancaiamao tiancaiamao merged commit 47526d3 into pingcap:master Aug 20, 2020
@tiancaiamao tiancaiamao deleted the table-reader-for-index-join branch August 20, 2020 02:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/executor status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

IndexMergeJoin on a partitioned table causes wrong result
5 participants