Skip to content

domain: load system keyspace info schema#61939

Merged
ti-chi-bot[bot] merged 21 commits into
pingcap:masterfrom
D3Hunter:load-system-keyspace-schema
Jun 25, 2025
Merged

domain: load system keyspace info schema#61939
ti-chi-bot[bot] merged 21 commits into
pingcap:masterfrom
D3Hunter:load-system-keyspace-schema

Conversation

@D3Hunter

@D3Hunter D3Hunter commented Jun 23, 2025

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: ref #61702

Problem Summary:

What changed and how does it work?

  • load info schema of system KS in user KS. we only do load, but not sync the system KS info schema, will sync it later. so we are still share the same schema validator with user KS
  • forbid access Domain inside cross keyspace session, add some nilness check inside the usage of Domain
  • make part of dependency of Domain in session as session level fields, such as InfoSchema, SchemaValidator and AutoIDRequirement
  • abstract some session creation logic
  • the Domain of the cross keyspace session is nil, as those sessions shouldn't access resources of current keyspace, to avoid it mix things up with user keyspace session and cause correct issues.
    • the cross keyspace session is only allowed to read/write to system tables, we have changed some of the code path that access Domain, if there are any other path we hasn't changed, let it panic, so we can know it early. we will sort out all other access point to Domain instance later to evaluate whether it's needed to change

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

as we don't have place to start 2 instance for SYSTEM and user KS, it's tested manually

apply below diff, and then start SYSTEM KS of nextgen, then start ks1 and ks2

the target=XXX means where the task is created, we can see ks1 and ks2 are using different id range for SYSTEM KS, and different with that of the current keyspace

on ks1 KS

["SYSTEM KEYSPACE TEST: created task"] [keyspaceName=ks1] [taskKey=ks1/0] [target="current keyspace"] [taskID=1]
["SYSTEM KEYSPACE TEST: created task"] [keyspaceName=ks1] [taskKey=ks1/0] [target="SYSTEM keyspace"] [taskID=1]
["SYSTEM KEYSPACE TEST: created task"] [keyspaceName=ks1] [taskKey=ks1/1] [target="current keyspace"] [taskID=2]
["SYSTEM KEYSPACE TEST: created task"] [keyspaceName=ks1] [taskKey=ks1/1] [target="SYSTEM keyspace"] [taskID=2]
["SYSTEM KEYSPACE TEST: created task"] [keyspaceName=ks1] [taskKey=ks1/2] [target="SYSTEM keyspace"] [taskID=3]
["SYSTEM KEYSPACE TEST: created task"] [keyspaceName=ks1] [taskKey=ks1/2] [target="current keyspace"] [taskID=3]

on ks2

["SYSTEM KEYSPACE TEST: created task"] [keyspaceName=ks2] [taskKey=ks2/0] [target="current keyspace"] [taskID=1]
["SYSTEM KEYSPACE TEST: created task"] [keyspaceName=ks2] [taskKey=ks2/0] [target="SYSTEM keyspace"] [taskID=30001]
["SYSTEM KEYSPACE TEST: created task"] [keyspaceName=ks2] [taskKey=ks2/1] [target="SYSTEM keyspace"] [taskID=30002]
["SYSTEM KEYSPACE TEST: created task"] [keyspaceName=ks2] [taskKey=ks2/1] [target="current keyspace"] [taskID=2]
["SYSTEM KEYSPACE TEST: created task"] [keyspaceName=ks2] [taskKey=ks2/2] [target="SYSTEM keyspace"] [taskID=30003]
["SYSTEM KEYSPACE TEST: created task"] [keyspaceName=ks2] [taskKey=ks2/2] [target="current keyspace"] [taskID=3]
["SYSTEM KEYSPACE TEST: created task"] [keyspaceName=ks2] [taskKey=ks2/3] [target="SYSTEM keyspace"] [taskID=30004]
["SYSTEM KEYSPACE TEST: created task"] [keyspaceName=ks2] [taskKey=ks2/3] [target="current keyspace"] [taskID=4]

diff

diff --git a/pkg/domain/domain.go b/pkg/domain/domain.go
index b30c76643c..063e93c385 100644
--- a/pkg/domain/domain.go
+++ b/pkg/domain/domain.go
@@ -108,6 +108,7 @@ import (
 	"github.com/pingcap/tidb/pkg/util/syncutil"
 	"github.com/pingcap/tidb/pkg/workloadlearning"
 	"github.com/tikv/client-go/v2/tikv"
+	tikvutil "github.com/tikv/client-go/v2/util"
 	pd "github.com/tikv/pd/client"
 	pdhttp "github.com/tikv/pd/client/http"
 	"github.com/tikv/pd/client/opt"
@@ -866,11 +867,33 @@ func (do *Domain) Start(startMode ddl.StartMode) error {
 		if err = do.loadSysKSInfoSchema(); err != nil {
 			return err
 		}
+		go func() {
+			time.Sleep(3 * time.Second)
+			go do.createDummyTasks(do.sysSessionPool, zap.String("target", "current keyspace"))
+			go do.createDummyTasks(do.sysksSessPool, zap.String("target", "SYSTEM keyspace"))
+		}()
 	}
 
 	return nil
 }
 
+func (do *Domain) createDummyTasks(sePool util.SessionPool, extraField zap.Field) {
+	taskMgr := storage.NewTaskManager(sePool)
+	ctx := tikvutil.WithInternalSourceType(context.Background(), kv.InternalDistTask)
+	for i := 0; i < 100; i++ {
+		taskKey := fmt.Sprintf("%s/%d", do.store.GetKeyspace(), i)
+		if taskID, err := taskMgr.CreateTask(ctx, taskKey, proto.TaskTypeExample,
+			1, "", 1, proto.ExtraParams{}, nil); err != nil {
+			logutil.BgLogger().Error("SYSTEM KEYSPACE TEST: failed to create task",
+				zap.String("taskKey", taskKey), extraField, zap.Error(err))
+		} else {
+			logutil.BgLogger().Info("SYSTEM KEYSPACE TEST: created task",
+				zap.String("taskKey", taskKey), extraField, zap.Int64("taskID", taskID))
+		}
+		time.Sleep(5 * time.Second)
+	}
+}
+
 // TODO: we should sync the system keyspace info schema, not just load it,
 // currently, we assume there is no upgrade, so we only load the info schema of
 // system keyspace once, and it will not change during the lifetime of the domain,
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. sig/planner SIG: Planner labels Jun 23, 2025
@tiprow

tiprow Bot commented Jun 23, 2025

Copy link
Copy Markdown

Hi @D3Hunter. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@codecov

codecov Bot commented Jun 23, 2025

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 56.68449% with 81 lines in your changes missing coverage. Please review.

Project coverage is 75.0061%. Comparing base (f701e5a) to head (8b0a371).
Report is 12 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #61939        +/-   ##
================================================
+ Coverage   72.9940%   75.0061%   +2.0121%     
================================================
  Files          1735       1781        +46     
  Lines        482030     493009     +10979     
================================================
+ Hits         351853     369787     +17934     
+ Misses       108602     100257      -8345     
- Partials      21575      22965      +1390     
Flag Coverage Δ
integration 49.2043% <48.1283%> (?)
unit 72.4272% <56.6844%> (+0.1871%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 52.7804% <ø> (ø)
parser ∅ <ø> (∅)
br 62.1230% <ø> (+15.5024%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@D3Hunter

Copy link
Copy Markdown
Contributor Author

/retest

@tiprow

tiprow Bot commented Jun 24, 2025

Copy link
Copy Markdown

@D3Hunter: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot ti-chi-bot Bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 24, 2025
@D3Hunter D3Hunter changed the title [WIP]domain: load system keyspace info schema domain: load system keyspace info schema Jun 24, 2025
@ti-chi-bot ti-chi-bot Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 24, 2025
Comment thread pkg/session/session.go
Comment on lines -1216 to -1240
se, err := createSession(store)
if err != nil {
return nil, err
}
err = se.sessionVars.SetSystemVar(vardef.AutoCommit, "1")
if err != nil {
return nil, err
}
err = se.sessionVars.SetSystemVar(vardef.MaxExecutionTime, "0")
if err != nil {
return nil, errors.Trace(err)
}
err = se.sessionVars.SetSystemVar(vardef.MaxAllowedPacket, strconv.FormatUint(vardef.DefMaxAllowedPacket, 10))
if err != nil {
return nil, errors.Trace(err)
}
err = se.sessionVars.SetSystemVar(vardef.TiDBConstraintCheckInPlacePessimistic, vardef.On)
if err != nil {
return nil, errors.Trace(err)
}
se.sessionVars.CommonGlobalLoaded = true
se.sessionVars.InRestrictedSQL = true
// Internal session uses default format to prevent memory leak problem.
se.sessionVars.EnableChunkRPC = false
return se, nil

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reuse what's in createSessionWithDomainFunc

@ti-chi-bot ti-chi-bot Bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Jun 25, 2025
@ti-chi-bot ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jun 25, 2025
@ti-chi-bot

ti-chi-bot Bot commented Jun 25, 2025

Copy link
Copy Markdown

[LGTM Timeline notifier]

Timeline:

  • 2025-06-25 07:22:48.126120002 +0000 UTC m=+861220.849298985: ☑️ agreed by tangenta.
  • 2025-06-25 08:40:55.37624715 +0000 UTC m=+865908.099426131: ☑️ agreed by GMHDBJD.

@ti-chi-bot

ti-chi-bot Bot commented Jun 25, 2025

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: GMHDBJD, hawkingrei, tangenta

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the approved label Jun 25, 2025
@D3Hunter

Copy link
Copy Markdown
Contributor Author

/retest

@tiprow

tiprow Bot commented Jun 25, 2025

Copy link
Copy Markdown

@D3Hunter: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@D3Hunter

Copy link
Copy Markdown
Contributor Author

/retest

@tiprow

tiprow Bot commented Jun 25, 2025

Copy link
Copy Markdown

@D3Hunter: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot ti-chi-bot Bot merged commit a268223 into pingcap:master Jun 25, 2025
27 checks passed
@D3Hunter D3Hunter deleted the load-system-keyspace-schema branch June 25, 2025 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm release-note-none Denotes a PR that doesn't merit a release note. sig/planner SIG: Planner size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants