br restore failed with error “type:UNKNOWN message:\"invalid store ID 0, not found\” when kill pd leader

[Lily2025]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

cluster deploy with 3 pd
1、br full restore
2、kill pdleader

2. What did you expect to see? (Required)

br restore can success

3. What did you see instead (Required)

br restore failed with error “type:UNKNOWN message:"invalid store ID 0, not found\” when kill pd leader

[collector.go:77] ["DataBase Restore failed summary"] [total-ranges=136] [ranges-succeed=135] [ranges-failed=1] [split-region=32.540336097s] [restore-ranges=1166] [unit-name=file] [error="type:UNKNOWN message:"invalid store ID 0, not found" ; type:UNKNOWN message:"invalid store ID 0, not found" ; type:UNKNOWN message:"invalid store ID 0, not found" ; type:UNKNOWN message:"invalid store ID 0, not found" "] [errorVerbose="the following errors occurred:\n - type:UNKNOWN message:"invalid store ID 0, not found" \n github.com/tikv/pd/client.(*client).respForErr\n \t/go/pkg/mod/github.com/tikv/pd/client@v0.0.0-20240126020320-567c7d43a008/client.go:1552\n github.com/tikv/pd/client.(*client).GetStore\n \t/go/pkg/mod/github.com/tikv/pd/client@v0.0.0-20240126020320-567c7d43a008/client.go:1122\n github.com/pingcap/tidb/br/pkg/restore/split.(*pdClient).GetStore\n \t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/restore/split/client.go:157\n github.com/pingcap/tidb/br/pkg/restore.(*importClient).GetImportClient\n \t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/restore/import.go:220\n github.com/pingcap/tidb/br/pkg/restore.(*importClient).MultiIngest\n \t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/restore/import.go:203\n github.com/pingcap/tidb/br/pkg/restore.(*FileImporter).ingestSSTs\n \t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/restore/import.go:1239\n github.com/pingcap/tidb/br/pkg/restore.(*FileImporter).ingest\n \t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/restore/import.go:1141\n github.com/pingcap/tidb/br/pkg/restore.(*FileImporter).ImportSSTFiles.func1\n \t/home/jenkins/agent/workspace/build-common/go/src/github.com/pingcap/br/br/pkg/restore/import.go:589\n github.com/pingcap/tidb/br/pkg/utils.WithRetry.func1

4. What is your TiDB version? (Required)

./tidb-server -V
Release Version: v8.0.0-alpha
Edition: Community
Git Commit Hash: 400bb2c
Git Branch: heads/refs/tags/v8.0.0-alpha
UTC Build Time: 2024-02-10 11:57:26
GoVersion: go1.21.6
Race Enabled: false
Check Table Before Drop: false
Store: unistore
2024-02-12T03:39:31.107+0800

[Lily2025]

/assign BornChanger

[BornChanger]

/assign @Leavrth

[BornChanger]

It's a common case at some special timing. please downgrade the severity to major. @jebter