BR - S3 Endpoint is not set when get bucket informations

[Lajule]

Bug Report

Hi,

We get following error when we try to backup our TiDB on s3 :

Error: s3 bucket and region are not matched, bucket=ringover-eu-2023, input region=fr-par, real region=us-east-1

After checking the code, i think it's because the endpoint is not set when calling GetBucketRegionWithClient here : https://github.com/pingcap/tidb/blob/master/br/pkg/storage/s3.go#L374

My guess is that you need a WithEndpoint when creating aws.Config there : https://github.com/pingcap/tidb/blob/master/br/pkg/storage/s3.go#L357

You can check this comment in AWS SDK : https://github.com/aws/aws-sdk-go/blob/main/service/s3/s3manager/bucket_region.go#L91

Use BR to backup DB

Try to backup with BR and specify a endpoint :

tiup br backup full --pd "$PDIP:$PDPORT" --s3.endpoint "https://s3.fr-par.scw.cloud/" -s "s3://bucket1/TEST1" --s3.region "fr-par" --send-credentials-to-tikv=true --ca "$CAFILE" --key "$KEYFILE" --cert "$CERTFILE"

This is the cli we use, the "bucket1" exists and the region fr-par is the good one, you can see the result of an aws cli call :

$ aws s3api get-bucket-location --bucket bucket1 --profile scw { "LocationConstraint": "fr-par" }

Backup is on S3

The backup fails

We get following :
Error: s3 bucket and region are not matched, bucket=ringover-eu-2023, input region=fr-par, real region=us-east-1

The region returned is not the same than the provided in cli because the endpoint is not set

TiDB 6.5

Everything works well before 6.5

regards

[Lajule]

We found the issue, we are sending a PR
regards,

[dveeden]

Let's keep the issue open until the PR has been merged.