*: implement CANCEL/DROP LOAD DATA JOB #42137

lance6716 · 2023-03-13T05:44:25Z

What problem does this PR solve?

Issue Number: ref #40499

Problem Summary:

What is changed and how it works?

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

Signed-off-by: lance6716 <lance6716@gmail.com>

ti-chi-bot · 2023-03-13T05:44:27Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

D3Hunter
GMHDBJD

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

Signed-off-by: lance6716 <lance6716@gmail.com>

ekexium · 2023-03-15T03:44:07Z

executor/insert_common.go

Changes in this file LGTM.

lance6716 · 2023-03-15T03:45:52Z

/cc @D3Hunter @GMHDBJD

GMHDBJD

LGTM

D3Hunter

rest lgtm

D3Hunter · 2023-03-15T07:38:35Z

executor/asyncloaddata/util.go

+	if !endTimeIsNull {
+		hasError := !rows[0].IsNull(2)
+		if hasError {
+			return exeerrors.ErrLoadDataInvalidOperation.GenWithStackByArgs("need status running or paused, but got failed")


make failed a status in job table?

currently we only have expected_status. but no actual_status, so these lines is to calculate actual_status from other columns. I plan to add actual_status later when implement PAUSE/RESUME LOAD DATA JOBS, to let PAUSE don't return until actual_status is paused 😂

D3Hunter · 2023-03-15T07:53:23Z

executor/load_data.go

@@ -642,10 +642,13 @@ func (e *LoadDataWorker) commitWork(ctx context.Context) (err error) {
 		tasks               uint64
 		lastScannedFileSize int64
 		currScannedFileSize int64
+		backgroundCtx       = context.Background()


err msg at line return errors.Errorf("failed to keepalive for LOAD DATA job %d", jobID) need updated too, job might be cancelled or deleted

24bd2b9
updated

Signed-off-by: lance6716 <lance6716@gmail.com>

GMHDBJD

LGTM

Signed-off-by: lance6716 <lance6716@gmail.com>

lance6716 · 2023-03-16T05:03:04Z

/retest

lance6716 · 2023-03-16T11:13:44Z

ping @D3Hunter

lance6716 · 2023-03-17T02:14:29Z

/merge

ti-chi-bot · 2023-03-17T02:14:32Z

This pull request has been accepted and is ready to merge.

Commit hash: 2f9da8d

Signed-off-by: lance6716 <lance6716@gmail.com>

lance6716 · 2023-03-17T02:42:00Z

/merge

ti-chi-bot · 2023-03-17T02:42:04Z

This pull request has been accepted and is ready to merge.

Commit hash: 76d23f7

lance6716 added 3 commits March 13, 2023 11:41

--wip-- [skip ci]

b1c25db

rename

c990190

Signed-off-by: lance6716 <lance6716@gmail.com>

--wip-- [skip ci]

6e2418c

ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 13, 2023

lance6716 added 8 commits March 13, 2023 22:40

Merge branch 'master' of github.com:pingcap/tidb into cancel-drop-ld

430838c

Signed-off-by: lance6716 <lance6716@gmail.com>

fix errdoc

78441b7

Signed-off-by: lance6716 <lance6716@gmail.com>

add some test

23af006

Signed-off-by: lance6716 <lance6716@gmail.com>

make bazel

dfe5cec

Signed-off-by: lance6716 <lance6716@gmail.com>

add missing file

2f5f576

--wip-- [skip ci]

1c4233c

fix some test

f2f8030

Signed-off-by: lance6716 <lance6716@gmail.com>

fix UT

f5c1643

Signed-off-by: lance6716 <lance6716@gmail.com>

ti-chi-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 14, 2023

log when error

27ec1c2

Signed-off-by: lance6716 <lance6716@gmail.com>

lance6716 changed the title ~~[WIP] *: implement CANCEL/DROP LOAD DATA JOB~~ *: implement CANCEL/DROP LOAD DATA JOB Mar 15, 2023

ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 15, 2023

add more test

e1eed46

Signed-off-by: lance6716 <lance6716@gmail.com>

ekexium reviewed Mar 15, 2023

View reviewed changes

executor/insert_common.go Outdated

Copy link

Member

ekexium Mar 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes in this file LGTM.

ti-chi-bot requested review from D3Hunter and GMHDBJD March 15, 2023 03:45

GMHDBJD approved these changes Mar 15, 2023

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Mar 15, 2023

D3Hunter reviewed Mar 15, 2023

View reviewed changes

lance6716 added 2 commits March 16, 2023 10:17

Merge branch 'master' of github.com:pingcap/tidb into cancel-drop-ld

c60933e

Signed-off-by: lance6716 <lance6716@gmail.com>

fix UT

eb94497

Signed-off-by: lance6716 <lance6716@gmail.com>

GMHDBJD approved these changes Mar 16, 2023

View reviewed changes

lance6716 added 2 commits March 16, 2023 11:32

fix another test

24bd2b9

Signed-off-by: lance6716 <lance6716@gmail.com>

fix another test

2f9da8d

Signed-off-by: lance6716 <lance6716@gmail.com>

D3Hunter approved these changes Mar 17, 2023

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Mar 17, 2023

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 17, 2023

lance6716 added 2 commits March 17, 2023 10:39

Merge branch 'master' of github.com:pingcap/tidb into cancel-drop-ld

05a97b2

fix unfound merge conflict

76d23f7

Signed-off-by: lance6716 <lance6716@gmail.com>

ti-chi-bot removed the status/can-merge Indicates a PR has been approved by a committer. label Mar 17, 2023

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 17, 2023

ti-chi-bot merged commit 04741ae into pingcap:master Mar 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: implement CANCEL/DROP LOAD DATA JOB #42137

*: implement CANCEL/DROP LOAD DATA JOB #42137

lance6716 commented Mar 13, 2023 •

edited

Loading

ti-chi-bot commented Mar 13, 2023 •

edited

Loading

ekexium Mar 15, 2023

lance6716 commented Mar 15, 2023

GMHDBJD left a comment

D3Hunter left a comment

D3Hunter Mar 15, 2023

lance6716 Mar 16, 2023

D3Hunter Mar 15, 2023

lance6716 Mar 16, 2023

GMHDBJD left a comment

lance6716 commented Mar 16, 2023

lance6716 commented Mar 16, 2023

lance6716 commented Mar 17, 2023

ti-chi-bot commented Mar 17, 2023

lance6716 commented Mar 17, 2023

ti-chi-bot commented Mar 17, 2023

*: implement CANCEL/DROP LOAD DATA JOB #42137

*: implement CANCEL/DROP LOAD DATA JOB #42137

Conversation

lance6716 commented Mar 13, 2023 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Mar 13, 2023 • edited Loading

ekexium Mar 15, 2023

Choose a reason for hiding this comment

lance6716 commented Mar 15, 2023

GMHDBJD left a comment

Choose a reason for hiding this comment

D3Hunter left a comment

Choose a reason for hiding this comment

D3Hunter Mar 15, 2023

Choose a reason for hiding this comment

lance6716 Mar 16, 2023

Choose a reason for hiding this comment

D3Hunter Mar 15, 2023

Choose a reason for hiding this comment

lance6716 Mar 16, 2023

Choose a reason for hiding this comment

GMHDBJD left a comment

Choose a reason for hiding this comment

lance6716 commented Mar 16, 2023

lance6716 commented Mar 16, 2023

lance6716 commented Mar 17, 2023

ti-chi-bot commented Mar 17, 2023

lance6716 commented Mar 17, 2023

ti-chi-bot commented Mar 17, 2023

lance6716 commented Mar 13, 2023 •

edited

Loading

ti-chi-bot commented Mar 13, 2023 •

edited

Loading