Dumpling will not retry upload to GCS if the server returned 503 #56127
Labels
affects-6.1
affects-6.5
affects-7.1
affects-7.5
affects-8.1
affects-8.3
component/dumpling
This is related to Dumpling of TiDB.
found/gs
found by gs
good first issue
Denotes an issue ready for a new contributor, according to the "help wanted" guidelines.
may-affects-5.4
This bug maybe affects 5.4.x versions.
report/customer
Customers have encountered this bug.
severity/major
type/bug
The issue is confirmed as a bug.
Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)
(I used mitmproxy to inject the 503 error. For testing there should be some zero-external-dependency means to do so 🤔 Also I used a local fake-gcs-server serving HTTP to avoid distractions of installing the self-signed TLS CA.)
Get mitmproxy
Prepare the following script, which will inject 503 for the first two requests to
*/o
(the URL for uploading objects to GCS)Run mitmproxy loaded with this script
Patch dumpling to use this proxy:
Run dumpling.
2. What did you expect to see? (Required)
Given that we only inject 503 twice, Dumpling should be able to successfully upload the file on its 3rd try and the whole process succeed.
Inside the mitmproxy console, we should be able to see two 503 responses like
3. What did you see instead (Required)
Dumpling failed without any retry, with logs like
4. What is your TiDB version? (Required)
Dumpling v8.3.0
The text was updated successfully, but these errors were encountered: