Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executor: fix wrong result for index merge read generate column #16002

Merged
merged 3 commits into from
Apr 14, 2020

Conversation

lzmhhh123
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #15994

Problem Summary: When an index merge reader reads the generated column, it gets the wrong result. Because the generated column is ignored in the final table reader stage.

What is changed and how it works?

What's Changed: deal the generated column for final table reader.

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Integration test

Side effects

  • Performance regression
    • Consumes more CPU

@lzmhhh123 lzmhhh123 added type/bugfix This PR fixes a bug. sig/execution SIG execution needs-cherry-pick-4.0 labels Apr 2, 2020
@lzmhhh123 lzmhhh123 requested a review from a team as a code owner April 2, 2020 05:43
@ghost ghost requested review from SunRunAway and removed request for a team April 2, 2020 05:43
@@ -189,6 +189,7 @@ func (e *IndexMergeReaderExecutor) startPartialIndexWorker(ctx context.Context,
SetKeepOrder(false).
SetStreaming(e.partialStreamings[workID]).
SetFromSessionVars(e.ctx.GetSessionVars()).
SetMemTracker(e.memTracker).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a bugfix?

Copy link
Contributor Author

@lzmhhh123 lzmhhh123 Apr 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably yes. It affects the memory track for index merge reader in the index side.

@codecov
Copy link

codecov bot commented Apr 2, 2020

Codecov Report

Merging #16002 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #16002   +/-   ##
===========================================
  Coverage   80.7645%   80.7645%           
===========================================
  Files           506        506           
  Lines        138198     138198           
===========================================
  Hits         111615     111615           
  Misses        18078      18078           
  Partials       8505       8505           

@@ -50,3 +50,14 @@ func (s *testSuite1) TestJoin(c *C) {
tk.MustQuery("select /*+ use_index_merge(t1, t1a, t1b) */ sum(t1.a) from t1 join t2 on t1.id = t2.id where t1.a < 2 or t1.b > 4").Check(testkit.Rows("6"))
tk.MustQuery("select /*+ use_index_merge(t1, t1a, t1b) */ sum(t1.a) from t1 join t2 on t1.id = t2.id where t1.a < 2 or t1.b > 5").Check(testkit.Rows("1"))
}

func (s *testSuite1) TestIndexMergeReaderAndGeneratedColumn(c *C) {
tk := testkit.NewTestKitWithInit(c, s.store)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to tk.MustExec("set tidb_enable_index_merge=1") ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the hint doesn't require tidb_enable_index_merge.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, but I think it's a bug.

If we do not enable_index_merge, the hint should not take effect

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think so.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to interfere, but the documentation currently mentions the following:

IndexMerge is disabled by default. Enable the IndexMerge in one of two ways:

  • Set the tidb_enable_index_merge system variable to 1;
  • Use the SQL Hint USE_INDEX_MERGE in the query.

Note: The SQL Hint has a higher priority over the system variable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright 😢

@lzmhhh123 lzmhhh123 requested review from zz-jason and XuHuaiyu April 10, 2020 04:49
Copy link
Contributor

@XuHuaiyu XuHuaiyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@francis0407 francis0407 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@francis0407 francis0407 added the status/can-merge Indicates a PR has been approved by a committer. label Apr 14, 2020
@sre-bot
Copy link
Contributor

sre-bot commented Apr 14, 2020

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Apr 14, 2020

@lzmhhh123 merge failed.

@lzmhhh123
Copy link
Contributor Author

/merge

@sre-bot
Copy link
Contributor

sre-bot commented Apr 14, 2020

Your auto merge job has been accepted, waiting for 16032, 16338

@sre-bot
Copy link
Contributor

sre-bot commented Apr 14, 2020

/run-all-tests

@sre-bot sre-bot merged commit c1a3170 into pingcap:master Apr 14, 2020
@lzmhhh123 lzmhhh123 deleted the bug-fix/index_merge_generate_column branch April 14, 2020 09:40
@sre-bot
Copy link
Contributor

sre-bot commented Apr 14, 2020

cherry pick to release-4.0 in PR #16359

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/execution SIG execution status/can-merge Indicates a PR has been approved by a committer. type/bugfix This PR fixes a bug.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

USE_INDEX_MERGE results in an incorrect result for a generated column
6 participants