planner: support index_lookup_merge_join in physical plan. #11338

lzmhhh123 · 2019-07-19T12:04:07Z

What problem does this PR solve?

A split pull request for #9571 .

What is changed and how it works?

Move the index_merge_join in planner.

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)

Code changes

Has exported function/method change
Has exported variable/fields change
Has interface methods change
Has persistent data change

Side effects

Possible performance regression
Increased code complexity

Related changes

None

codecov · 2019-07-19T12:09:52Z

Codecov Report

Merging #11338 into master will decrease coverage by 0.2554%.
The diff coverage is 100%.

@@               Coverage Diff                @@
##             master     #11338        +/-   ##
================================================
- Coverage   81.5939%   81.3384%   -0.2555%     
================================================
  Files           445        445                
  Lines         96566      95571       -995     
================================================
- Hits          78792      77736      -1056     
- Misses        12241      12302        +61     
  Partials       5533       5533

planner/core/physical_plans.go

planner/core/stringer.go

planner/core/exhaust_physical_plans.go

planner/core/task.go

planner/core/exhaust_physical_plans.go

…erge_join_in_physical_plan

eurekaka

LGTM

planner/core/task.go

Co-Authored-By: Kenan Yao <cauchy1992@gmail.com>

zz-jason · 2019-08-24T09:13:13Z

cmd/explaintest/r/topn_push_down.result

-    └─IndexReader_91	9.99	root	index:Selection_90
-      └─Selection_90	9.99	cop	not(isnull(test.p.relate_id))
-        └─IndexScan_89	10.00	cop	table:p, index:relate_id, range: decided by [eq(test.p.relate_id, test.tr.id)], keep order:false, stats:pseudo
+└─TopN_16	0.00	root	test.te.expect_time:asc, offset:0, count:5


It's wried, it should be a Limit operator?

It's the result of rule_topn_push_down.

sre-bot · 2019-08-28T03:04:33Z

@zz-jason, @qw4990, @lamxTyler, @eurekaka, @winoros, PTAL.

sre-bot · 2019-08-30T03:04:33Z

@zz-jason, @qw4990, @lamxTyler, @eurekaka, @winoros, PTAL.

…erge_join_in_physical_plan

sre-bot · 2019-09-02T03:04:33Z

@zz-jason, @qw4990, @lamxTyler, @eurekaka, @winoros, PTAL.

zz-jason · 2019-09-02T08:39:00Z

planner/core/initialize.go

@@ -404,6 +406,15 @@ func (p PhysicalIndexJoin) Init(ctx sessionctx.Context, stats *property.StatsInf
 	return &p
 }

+// Init initializes PhysicalIndexMergeJoin.
+func (p PhysicalIndexMergeJoin) Init(ctx sessionctx.Context) *PhysicalIndexMergeJoin {
+	ctx.GetSessionVars().PlanID++


We can call p.PhysicalIndexJoin.Init(ctx), then reset p.tp.

PhysicalIndexJoin.Init(ctx) news a new basePlan. I think that's unnecessary.

zz-jason · 2019-09-02T08:40:22Z

planner/core/physical_plan_test.go

@@ -1360,20 +1360,20 @@ func (s *testPlanSuite) TestIndexJoinUnionScan(c *C) {
 	}{
 		// Test Index Join + UnionScan + TableScan.
 		{
-			sql:  "select /*+ TIDB_INLJ(t2) */ * from t t1, t t2 where t1.a = t2.a",
-			best: "IndexJoin{TableReader(Table(t))->UnionScan([])->TableReader(Table(t))->UnionScan([])}(test.t1.a,test.t2.a)",
+			sql:  "select /*+ TIDB_INLJ(t1, t2) */ * from t t1, t t2 where t1.a = t2.a",


why changing the hint?

…erge_join_in_physical_plan

log.txt

alivxxx

LGTM

sre-bot · 2019-09-04T05:41:34Z

/run-all-tests

support index_lookup_merge_join in physical plan

d647ff5

lzmhhh123 added type/enhancement The issue or PR belongs to an enhancement. sig/planner SIG: Planner type/new-feature labels Jul 19, 2019

lzmhhh123 requested review from eurekaka, XuHuaiyu and zz-jason July 19, 2019 12:26

zz-jason reviewed Jul 23, 2019

View reviewed changes

planner/core/physical_plans.go Outdated Show resolved Hide resolved

planner/core/stringer.go Show resolved Hide resolved

planner/core/exhaust_physical_plans.go Outdated Show resolved Hide resolved

qw4990 self-requested a review July 24, 2019 08:26

lzmhhh123 added 3 commits July 31, 2019 19:07

Merge branch 'master' into dev/support_index_merge_join_in_physical_plan

3b69d4b

fix

535f944

fix explain test

38a0de2

lzmhhh123 requested review from winoros and zz-jason August 1, 2019 02:54

lzmhhh123 force-pushed the dev/support_index_merge_join_in_physical_plan branch from e6cf850 to 38a0de2 Compare August 1, 2019 05:00

zz-jason reviewed Aug 1, 2019

View reviewed changes

qw4990 reviewed Aug 1, 2019

View reviewed changes

planner/core/task.go Outdated Show resolved Hide resolved

qw4990 reviewed Aug 1, 2019

View reviewed changes

planner/core/exhaust_physical_plans.go Show resolved Hide resolved

lzmhhh123 added 2 commits August 1, 2019 15:12

address comments

d68dcb0

add comments for NeedOuterSort, CompareFuncs and OuterCompareFuncs

05b6b5b

lzmhhh123 force-pushed the dev/support_index_merge_join_in_physical_plan branch from e64ebc4 to 05b6b5b Compare August 1, 2019 07:21

zz-jason requested review from zz-jason and qw4990 August 5, 2019 08:53

zz-jason added the status/PTAL label Aug 5, 2019

alivxxx reviewed Aug 8, 2019

View reviewed changes

planner/core/exhaust_physical_plans.go Outdated Show resolved Hide resolved

planner/core/exhaust_physical_plans.go Outdated Show resolved Hide resolved

lzmhhh123 added status/ReqChange and removed status/PTAL labels Aug 8, 2019

Merge remote-tracking branch 'origin/master' into dev/support_index_m…

8ed2c1d

…erge_join_in_physical_plan

zz-jason removed their request for review August 8, 2019 06:19

eurekaka reviewed Aug 23, 2019

View reviewed changes

planner/core/task.go Outdated Show resolved Hide resolved

planner/core/task.go Outdated Show resolved Hide resolved

planner/core/task.go Outdated Show resolved Hide resolved

lzmhhh123 and others added 3 commits August 23, 2019 17:42

commit suggestion

55bd051

Co-Authored-By: Kenan Yao <cauchy1992@gmail.com>

commit suggestion

f8c5f9a

Co-Authored-By: Kenan Yao <cauchy1992@gmail.com>

commit suggestion

029cc86

Co-Authored-By: Kenan Yao <cauchy1992@gmail.com>

lzmhhh123 added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 23, 2019

zz-jason reviewed Aug 24, 2019

View reviewed changes

lzmhhh123 added 2 commits August 30, 2019 15:41

fix

0ad1387

Merge remote-tracking branch 'origin/master' into dev/support_index_m…

5b5200b

…erge_join_in_physical_plan

lzmhhh123 requested a review from zz-jason August 30, 2019 08:24

qw4990 removed their request for review September 2, 2019 08:37

zz-jason reviewed Sep 2, 2019

View reviewed changes

XuHuaiyu mentioned this pull request Sep 3, 2019

Refine Index Join #8470

Closed

Merge remote-tracking branch 'origin/master' into dev/support_index_m…

45dacec

…erge_join_in_physical_plan

lzmhhh123 requested review from zz-jason and alivxxx September 3, 2019 07:07

alivxxx reviewed Sep 4, 2019

View reviewed changes

log.txt Outdated Show resolved Hide resolved

rm log.txt

3d7dc49

lzmhhh123 requested a review from alivxxx September 4, 2019 03:24

Merge branch 'master' into dev/support_index_merge_join_in_physical_plan

3927127

alivxxx approved these changes Sep 4, 2019

View reviewed changes

alivxxx added status/LGT2 Indicates that a PR has LGTM 2. status/can-merge Indicates a PR has been approved by a committer. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Sep 4, 2019

sre-bot merged commit 8450613 into pingcap:master Sep 4, 2019

lzmhhh123 deleted the dev/support_index_merge_join_in_physical_plan branch September 4, 2019 05:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

planner: support index_lookup_merge_join in physical plan. #11338

planner: support index_lookup_merge_join in physical plan. #11338

lzmhhh123 commented Jul 19, 2019 •

edited

Loading

codecov bot commented Jul 19, 2019 •

edited

Loading

eurekaka left a comment

zz-jason Aug 24, 2019

lzmhhh123 Aug 26, 2019

sre-bot commented Aug 28, 2019

sre-bot commented Aug 30, 2019

sre-bot commented Sep 2, 2019

zz-jason Sep 2, 2019

lzmhhh123 Sep 2, 2019

zz-jason Sep 2, 2019

lzmhhh123 Sep 3, 2019

alivxxx left a comment

sre-bot commented Sep 4, 2019

planner: support index_lookup_merge_join in physical plan. #11338

planner: support index_lookup_merge_join in physical plan. #11338

Conversation

lzmhhh123 commented Jul 19, 2019 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

codecov bot commented Jul 19, 2019 • edited Loading

Codecov Report

eurekaka left a comment

Choose a reason for hiding this comment

zz-jason Aug 24, 2019

Choose a reason for hiding this comment

lzmhhh123 Aug 26, 2019

Choose a reason for hiding this comment

sre-bot commented Aug 28, 2019

sre-bot commented Aug 30, 2019

sre-bot commented Sep 2, 2019

zz-jason Sep 2, 2019

Choose a reason for hiding this comment

lzmhhh123 Sep 2, 2019

Choose a reason for hiding this comment

zz-jason Sep 2, 2019

Choose a reason for hiding this comment

lzmhhh123 Sep 3, 2019

Choose a reason for hiding this comment

alivxxx left a comment

Choose a reason for hiding this comment

sre-bot commented Sep 4, 2019

lzmhhh123 commented Jul 19, 2019 •

edited

Loading

codecov bot commented Jul 19, 2019 •

edited

Loading