-
Notifications
You must be signed in to change notification settings - Fork 253
tikv: close region cache before PD client #1853
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
|
Welcome @fzzf678! |
📝 WalkthroughWalkthroughReorders KVStore shutdown: txn latches closed earlier, region cache closed before PD clients (with a failpoint-checked runtime panic if not), PD HTTP then PD gRPC clients closed, then TiKV clients. Adds RegionCache.IsBackgroundRunnerClosed and a test validating close ordering. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant KV as KVStore
participant TL as TxnLatches
participant RC as RegionCache
participant PHTTP as PD_HTTP_Client
participant PGRPC as PD_gRPC_Client
participant TK as TiKV_Client
rect rgba(200,230,255,0.5)
KV->>TL: Close() (if present)
TL-->>KV: closed
end
rect rgba(200,255,200,0.5)
KV->>RC: Close() (cancel background runner)
RC-->>KV: background goroutines stopped
end
rect rgba(255,230,200,0.5)
KV->>PHTTP: Close()
PHTTP-->>KV: closed
KV->>PGRPC: Close()
PGRPC-->>KV: closed
end
rect rgba(240,200,255,0.5)
KV->>TK: Close()
TK-->>KV: closed
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Signed-off-by: fzzf678 <daery_7@163.com>
58eae61 to
3b25743
Compare
tikv/kv.go
Outdated
| } | ||
| s.regionCache.Close() | ||
| if _, err := util.EvalFailpoint("checkRegionCacheClosedBeforePDClose"); err == nil && retErr == nil && !s.regionCache.IsBackgroundRunnerClosed() { | ||
| retErr = errors.New("region cache is not closed before closing pd client") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we panic here? seems easier to understand and the panic in failpoint does not affect the production code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Of course
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
update in 62192c0
[LGTM Timeline notifier]Timeline:
|
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: lcwangchao The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Please repair the DCO with |
62192c0 to
17efe31
Compare
tikv: add regression test for KVStore.Close order Revert "tikv: add regression test for KVStore.Close order" This reverts commit 40b5491714d16650a9aa73e8d5a8e2313df9656c. Signed-off-by: fzzf678 <daery_7@163.com>
Signed-off-by: fzzf678 <daery_7@163.com>
17efe31 to
a45885f
Compare
Fixes #1852.
KVStore.Close() previously closed the PD client before closing RegionCache. RegionCache has background tasks (store list refresh / store re-resolve) that may still call PD APIs (GetStore/GetAllStores). When PD client is closed first, those background calls fail and emit noisy logs like:
This PR reorders KVStore.Close() to close RegionCache (and other dependents) before closing the PD clients, avoiding the shutdown window error.
Summary by CodeRabbit
Chores
Public API
Tests
✏️ Tip: You can customize this high-level summary in your review settings.