Retry following redirects on 400 (not 404), 502, 503, & 504 up to 5 times

[jwodder]

Closes #761.

[codecov]

Codecov Report

Merging #762 (e09c864) into master (ed67c1d) will increase coverage by 0.03%.
The diff coverage is 50.00%.

@@            Coverage Diff             @@
##           master     #762      +/-   ##
==========================================
+ Coverage   84.74%   84.78%   +0.03%     
==========================================
  Files          59       59              
  Lines        6026     6027       +1     
==========================================
+ Hits         5107     5110       +3     
+ Misses        919      917       -2     

Flag	Coverage Δ
unittests	84.78% <50.00%> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
dandi/dandiarchive.py	82.23% <50.00%> (-1.53%)	⬇️
dandi/tests/test_download.py	100.00% <0.00%> (ø)
dandi/pynwb_utils.py	82.45% <0.00%> (+0.10%)	⬆️
dandi/_version.py	45.96% <0.00%> (+1.75%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ed67c1d...e09c864. Read the comment docs.

[yarikoptic]

I guess a unittest needs fix ups to reflect 404 -> 400 change

________________________ test_parse_dandi_url_not_found ________________________
dandi/tests/test_dandiarchive.py:240: in test_parse_dandi_url_not_found
    parse_dandi_url("https://dandiarchive.org/dandiset/999999")
dandi/dandiarchive.py:421: in parse
    new_url = cls.follow_redirect(url)
dandi/dandiarchive.py:537: in follow_redirect
    raise FailedToConnectError(
E   dandi.exceptions.FailedToConnectError: Response for getting https://dandiarchive.org/dandiset/999999 to redirect returned 404. Please verify that it is a URL related to dandiarchive and supported by dandi client

[jwodder]

@yarikoptic Fixed.

[yarikoptic]

Great, thank you!