BR tool ErrRestoreTableIDMismatch error when using raw kv mode

[harshul4274]

1. Minimal reproduce step (Required)

Hello Everyone,

We have used pingcap TiDB Operator to deploy TiKV cluster in Kubernetes. We want to use TiKV only not TiDB, so we have disabled TiDB component.
We want to take backup of our TiKV data so we have setup one pod which has BR tool installed.
We are using below commands to take backup and restore it with raw mode:

Backup:
./br backup raw --pd "test-tikv-pd.tikv:2379"
--storage "s3://tikv/backup"
--s3.region "us-west-2"
--send-credentials-to-tikv=true
--start 1
--end zzzzzzzz
--ratelimit 128
--format raw
--cf default
--log-file backupfull.log

Restore:
./br restore raw --pd "test-tikv-pd.tikv:2379"
--storage "s3://tikv/backup"
--s3.region "us-west-2"
--send-credentials-to-tikv=true
--start 1
--end zzzzzzzz
--ratelimit 128
--format raw
--cf default
--log-file restore_raw.log

2. What did you expect to see? (Required)

Ideally backup and restore should work with any key range in raw kv mode

3. What did you see instead (Required)

When we take backup using BR tool using above commands, backup works fine and we can see data also into our AWS S3 storage bucket, but when we try to restore that data using BR tool, it is giving below error:

Error: table id mismatch: [BR:Restore:ErrRestoreTableIDMismatch]restore table ID mismatch

After working with team on Slack (https://tidbcommunity.slack.com/archives/CH7TTLL7P/p1654330656688959?thread_ts=1653049435.952779&cid=CH7TTLL7P), we found that we can not take backup of keys that are starting with "t" because if we take backup of that keys (which are internal for TiDB) then at the time of restore it is giving error as mentioned above.

So for example below backup and restore command worked for us:

Backup:
./br backup raw --pd "test-tikv-pd.tikv:2379"
--storage "s3://tikv/backup"
--s3.region "us-west-2"
--send-credentials-to-tikv=true
--start k
--end l
--ratelimit 128
--format raw
--cf default
--log-file backupfull.log

Restore:
./br restore raw --pd "test-tikv-pd.tikv:2379"
--storage "s3://tikv/backup"
--s3.region "us-west-2"
--send-credentials-to-tikv=true
--start k
--end l
--ratelimit 128
--format raw
--cf default
--log-file restore_raw.log

But this workaround is not ideal because we cannot use keys starting with "t".

4. What is your TiDB version? (Required)

We are using TiDB Operator 1.3.3 and TiKV 5.4.1