Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

executor: fix issue of load data statement doesn't record into slow query and statements_summary #20713

Merged
merged 15 commits into from
Nov 4, 2020
Merged

executor: fix issue of load data statement doesn't record into slow query and statements_summary #20713

merged 15 commits into from
Nov 4, 2020

Conversation

crazycs520
Copy link
Contributor

What problem does this PR solve?

close #20553

Before this PR, LOAD DATA statement will not rightly record into SLOW_QUERY and STATEMENTS_SUMMARY, since LOAD DATA statement is a special query, it's processing flow is different from nomal query (like insert/delete).

This PR will fix this problem.

Here is an example in this PR:

information_schema> select query_time,plan from information_schema.slow_query where query like 'load data %' order by time desc limit 1;
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| query_time  | plan                                                                                                                                                                                                                                                                                                                                                             |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 0.894166186 |     id             task    estRows    operator info    actRows    execution info                                                                                                                                                                                                                                                                  memory    disk |
|             |     Load_Data_1    root    0          N/A              0          time:419.848044ms, loops:2, prepare:6.87µs, check_insert:{total_time:419.838734ms, mem_insert_time:418.726803ms, prefetch:1.111931ms}, commit_txn: {prewrite:284.478315ms, get_commit_ts:4.46µs, commit:34.507968ms, region_num:1, write_keys:424356, write_byte:15809789}    N/A       N/A    |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set
Time: 0.017s
information_schema> select sum_latency,plan from `STATEMENTS_SUMMARY` where QUERY_SAMPLE_TEXT like "load data%%" limit 1;
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| sum_latency | plan                                                                                                                                                                                                                                                                                                                                                             |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 894456516   |     id             task    estRows    operator info    actRows    execution info                                                                                                                                                                                                                                                                  memory    disk |
|             |     Load_Data_1    root    0          N/A              0          time:419.848044ms, loops:2, prepare:6.87µs, check_insert:{total_time:419.838734ms, mem_insert_time:418.726803ms, prefetch:1.111931ms}, commit_txn: {prewrite:284.478315ms, get_commit_ts:4.46µs, commit:34.507968ms, region_num:1, write_keys:424356, write_byte:15809789}    N/A       N/A    |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

What is changed and how it works?

Record SLOW_QUERY and STATEMENTS_SUMMARY after handled the special query.

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Manual test

Side effects

  • Performance regression
    • Consumes more CPU

Release note

  • Fix issue of load data statement doesn't record into slow query and statements_summary

@crazycs520
init
2edfd98 
Signed-off-by: crazycs520 <crazycs520@gmail.com>
@crazycs520
update
d837256 
Signed-off-by: crazycs520 <crazycs520@gmail.com>
@crazycs520
refine code
d015b57 
Signed-off-by: crazycs520 <crazycs520@gmail.com>
@crazycs520
add test
0b8669a 
Signed-off-by: crazycs520 <crazycs520@gmail.com>
@crazycs520
refine test
9aa4205 
Signed-off-by: crazycs520 <crazycs520@gmail.com>
@crazycs520 crazycs520 added the type/bugfix This PR fixes a bug. label Oct 29, 2020
@crazycs520 crazycs520 requested review from a team as code owners October 29, 2020 07:27
@crazycs520 crazycs520 requested review from XuHuaiyu and removed request for a team October 29, 2020 07:27
@github-actions github-actions bot added sig/execution SIG execution sig/planner SIG: Planner labels Oct 29, 2020
@crazycs520
Copy link
Contributor Author

/run-all-tests

1 similar comment
@crazycs520
Copy link
Contributor Author

/run-all-tests

@crazycs520 crazycs520 mentioned this pull request Oct 29, 2020
Open
25 tasks
wjhuang2016
wjhuang2016 reviewed Nov 2, 2020
View reviewed changes
executor/load_data.go Outdated Show resolved Hide resolved
util/plancodec/id.go Outdated Show resolved Hide resolved
@tiancaiamao
Copy link
Contributor

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Nov 2, 2020
@ichn-hu ichn-hu mentioned this pull request Nov 3, 2020
Closed
@crazycs520
address comment
1ca0c1d 
Signed-off-by: crazycs520 <crazycs520@gmail.com>
@crazycs520
Merge branch 'master' into load-data-stats
896fd7f
wjhuang2016
wjhuang2016 approved these changes Nov 4, 2020
View reviewed changes
Copy link
Member

@wjhuang2016 wjhuang2016 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot removed the status/LGT1 Indicates that a PR has LGTM 1. label Nov 4, 2020
@ti-srebot ti-srebot added the status/LGT2 Indicates that a PR has LGTM 2. label Nov 4, 2020
@crazycs520
Copy link
Contributor Author

/run-all-tests

@crazycs520
Copy link
Contributor Author

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Nov 4, 2020
@ti-srebot
Copy link
Contributor

Your auto merge job has been accepted, waiting for:

  • 20824

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot ti-srebot merged commit ae5dc3f into pingcap:master Nov 4, 2020
ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Nov 4, 2020
@crazycs520 @ti-srebot
cherry pick pingcap#20713 to release-4.0
68d3cb3 
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot ti-srebot mentioned this pull request Nov 4, 2020
Merged
@ti-srebot
Copy link
Contributor

cherry pick to release-4.0 in PR #20830

crazycs520 added a commit to ti-srebot/tidb that referenced this pull request Nov 4, 2020
@crazycs520
Revert "cherry pick pingcap#20713 to release-4.0"
b38eaff 
This reverts commit 68d3cb3.
crazycs520 added a commit to ti-srebot/tidb that referenced this pull request Nov 4, 2020
@crazycs520
executor: fix issue of load data statement doesn't record into slow q…
f9f67d6 
…uery and statements_summary (pingcap#20713)

Signed-off-by: crazycs520 <crazycs520@gmail.com>
ti-srebot added a commit that referenced this pull request Nov 13, 2020
@ti-srebot
executor: fix issue of load data statement doesn't record into slow q…
acb948c 
…uery and statements_summary (#20713) (#20830)

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
Signed-off-by: crazycs520 <crazycs520@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tiancaiamao tiancaiamao tiancaiamao left review comments

@ti-srebot ti-srebot ti-srebot approved these changes

@wjhuang2016 wjhuang2016 wjhuang2016 approved these changes

@qw4990 qw4990 Awaiting requested review from qw4990

@XuHuaiyu XuHuaiyu Awaiting requested review from XuHuaiyu

Assignees
No one assigned
Labels
sig/execution SIG execution sig/planner SIG: Planner status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. type/bugfix This PR fixes a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

load data statement doesn't record into slow-log and statement
4 participants
@crazycs520 @tiancaiamao @ti-srebot @wjhuang2016