lightning: mark background and resource group name when in local mode #44950

HuSharp · 2023-06-26T07:58:37Z

Signed-off-by: husharp jinhao.hu@pingcap.com

What problem does this PR solve?

Issue Number: ref #44517
wait for tikv/client-go#872 merged.

Problem Summary:

What is changed and how it works?

Check List

Tests

Manual test (add detailed scripts or steps below)

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

ti-chi-bot · 2023-06-26T07:58:40Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

tiprow · 2023-06-26T07:58:57Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

HuSharp · 2023-07-03T08:47:05Z

@glorv @nolouch PTAL, thx!

glorv

The resource-group part LGTM. Should also add resource-group-tag so tikv can identify the task type. /cc @nolouch @lance6716

br/pkg/lightning/importer/import.go

br/cmd/tidb-lightning-ctl/main.go

br/pkg/lightning/importer/import.go

go.mod

HuSharp · 2023-07-26T12:17:34Z

/retest-required

HuSharp · 2023-07-27T01:58:08Z

/retest-required

glorv · 2023-07-27T03:50:44Z

/retest

HuSharp · 2023-07-27T05:25:42Z

/retest-required

glorv · 2023-07-27T07:12:18Z

@lance6716 @nolouch PTAL again, thanks

lance6716

rest lgtm

lance6716 · 2023-07-27T07:26:26Z

br/pkg/lightning/importer/checksum_helper.go

@@ -60,7 +60,13 @@ func NewChecksumManager(ctx context.Context, rc *Controller, store kv.Storage) (
 			log.FromContext(ctx).Info("set tidb_backoff_weight to default", zap.Int("backoff_weight", local.DefaultBackoffWeight))
 			backoffWeight = local.DefaultBackoffWeight
 		}
-		manager = local.NewTiKVChecksumManager(store.GetClient(), pdCli, uint(rc.cfg.TiDB.DistSQLScanConcurrency), backoffWeight)
+
+		explicitRequestSourceType, err := common.GetExplicitRequestSourceTypeFromDB(ctx, rc.db)


Will it fail when the target cluster is lower version? Is this by design?

Querying a lower version of tidb only returns empty results.

When getting empty results, it will be the same as the default setting.

br/pkg/lightning/importer/import.go

lance6716 · 2023-07-27T07:29:39Z

ddl/ingest/backend_mgr.go

@@ -121,7 +121,7 @@ func createLocalBackend(ctx context.Context, cfg *Config) (*local.Backend, error
 	regionSizeGetter := &local.TableRegionSizeGetterImpl{
 		DB: nil,
 	}
-	backendConfig := local.NewBackendConfig(cfg.Lightning, int(LitRLimit), cfg.KeyspaceName)
+	backendConfig := local.NewBackendConfig(cfg.Lightning, int(LitRLimit), cfg.KeyspaceName, "")


maybe ask for a DDL developer to review the value

@wjhuang2016 PTAL

Signed-off-by: husharp <jinhao.hu@pingcap.com>

lance6716 · 2023-07-27T08:18:36Z

lgtm

HuSharp · 2023-07-27T08:47:39Z

/retest-required

HuSharp · 2023-07-27T09:25:14Z

/retest-required

Signed-off-by: husharp <jinhao.hu@pingcap.com>

HuSharp · 2023-07-28T05:24:29Z

/retest-required

ti-chi-bot · 2023-07-28T06:04:51Z

@HuSharp: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-br-integration-test	`55cc1dc`	link	true	`/test pull-br-integration-test`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

HuSharp · 2023-07-28T07:19:18Z

/retest-required

ti-chi-bot · 2023-07-28T10:48:38Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: glorv, tangenta

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [glorv,tangenta]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2023-07-28T10:48:38Z

[LGTM Timeline notifier]

Timeline:

2023-07-26 05:04:04.283504413 +0000 UTC m=+330786.878035400: ☑️ agreed by glorv.
2023-07-28 10:48:37.468365413 +0000 UTC m=+524260.062896400: ☑️ agreed by tangenta.

ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 26, 2023

HuSharp force-pushed the lightning_bind_resource_group branch from 83990ad to 5b4d006 Compare June 30, 2023 08:51

ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jun 30, 2023

HuSharp force-pushed the lightning_bind_resource_group branch 4 times, most recently from 65c75b2 to 48532a6 Compare July 3, 2023 07:59

HuSharp marked this pull request as ready for review July 3, 2023 08:14

ti-chi-bot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 3, 2023

ti-chi-bot bot deleted a comment from ti-chi-bot Jul 3, 2023

HuSharp force-pushed the lightning_bind_resource_group branch from 48532a6 to c9f2cc0 Compare July 3, 2023 08:27

ti-chi-bot bot deleted a comment from ti-chi-bot Jul 3, 2023

glorv reviewed Jul 3, 2023

View reviewed changes

br/pkg/lightning/importer/import.go Outdated Show resolved Hide resolved

br/cmd/tidb-lightning-ctl/main.go Outdated Show resolved Hide resolved

glorv requested review from lance6716 and nolouch July 3, 2023 09:13

lance6716 reviewed Jul 3, 2023

View reviewed changes

br/pkg/lightning/importer/import.go Outdated Show resolved Hide resolved

go.mod Outdated Show resolved Hide resolved

HuSharp force-pushed the lightning_bind_resource_group branch from c9f2cc0 to fc3e0bc Compare July 4, 2023 09:01

HuSharp requested a review from a team as a code owner July 4, 2023 09:01

ti-chi-bot bot deleted a comment from ti-chi-bot Jul 4, 2023

lance6716 reviewed Jul 27, 2023

View reviewed changes

rename IsFunctionNotExistErr

4dccba1

Signed-off-by: husharp <jinhao.hu@pingcap.com>

glorv removed the request for review from a team July 28, 2023 02:58

ti-chi-bot bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 28, 2023

merge master

7c0a646

Signed-off-by: husharp <jinhao.hu@pingcap.com>

merge master

2ca33d8

Signed-off-by: husharp <jinhao.hu@pingcap.com>

HuSharp force-pushed the lightning_bind_resource_group branch from b6f06cd to 2ca33d8 Compare July 28, 2023 04:47

tangenta approved these changes Jul 28, 2023

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 28, 2023

ti-chi-bot bot merged commit f4f78a1 into pingcap:master Jul 28, 2023
10 of 13 checks passed

HuSharp deleted the lightning_bind_resource_group branch July 28, 2023 11:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lightning: mark background and resource group name when in local mode #44950

lightning: mark background and resource group name when in local mode #44950

HuSharp commented Jun 26, 2023 •

edited by glorv

Loading

ti-chi-bot bot commented Jun 26, 2023

tiprow bot commented Jun 26, 2023

HuSharp commented Jul 3, 2023

glorv left a comment

HuSharp commented Jul 26, 2023

HuSharp commented Jul 27, 2023

glorv commented Jul 27, 2023

HuSharp commented Jul 27, 2023

glorv commented Jul 27, 2023

lance6716 left a comment

lance6716 Jul 27, 2023

HuSharp Jul 27, 2023

lance6716 Jul 27, 2023

glorv Jul 27, 2023 •

edited

Loading

tangenta Jul 28, 2023

lance6716 commented Jul 27, 2023

HuSharp commented Jul 27, 2023

HuSharp commented Jul 27, 2023

HuSharp commented Jul 28, 2023

ti-chi-bot bot commented Jul 28, 2023 •

edited

Loading

HuSharp commented Jul 28, 2023

ti-chi-bot bot commented Jul 28, 2023

ti-chi-bot bot commented Jul 28, 2023

lightning: mark background and resource group name when in local mode #44950

lightning: mark background and resource group name when in local mode #44950

Conversation

HuSharp commented Jun 26, 2023 • edited by glorv Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot bot commented Jun 26, 2023

tiprow bot commented Jun 26, 2023

HuSharp commented Jul 3, 2023

glorv left a comment

Choose a reason for hiding this comment

HuSharp commented Jul 26, 2023

HuSharp commented Jul 27, 2023

glorv commented Jul 27, 2023

HuSharp commented Jul 27, 2023

glorv commented Jul 27, 2023

lance6716 left a comment

Choose a reason for hiding this comment

lance6716 Jul 27, 2023

Choose a reason for hiding this comment

HuSharp Jul 27, 2023

Choose a reason for hiding this comment

lance6716 Jul 27, 2023

Choose a reason for hiding this comment

glorv Jul 27, 2023 • edited Loading

Choose a reason for hiding this comment

tangenta Jul 28, 2023

Choose a reason for hiding this comment

lance6716 commented Jul 27, 2023

HuSharp commented Jul 27, 2023

HuSharp commented Jul 27, 2023

HuSharp commented Jul 28, 2023

ti-chi-bot bot commented Jul 28, 2023 • edited Loading

HuSharp commented Jul 28, 2023

ti-chi-bot bot commented Jul 28, 2023

ti-chi-bot bot commented Jul 28, 2023

[LGTM Timeline notifier]

HuSharp commented Jun 26, 2023 •

edited by glorv

Loading

glorv Jul 27, 2023 •

edited

Loading

ti-chi-bot bot commented Jul 28, 2023 •

edited

Loading