fix: guard against anomalous crt API responses#3077
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #3077 +/- ##
=====================================
+ Coverage 91% 91% +1%
=====================================
Files 437 437
Lines 37509 37511 +2
=====================================
+ Hits 33925 33927 +2
Misses 3584 3584 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Thanks for catching this. The 404/503 handling is definitely a real gap. However as written this short-circuits the existing api retry logic:
return getattr(r, "is_success", False) or getattr(r, "status_code", 0) not in {404, 503}The second clause is True for any status code other than 404/503, so 429, 500, 502, etc. all get marked as success and skip the retry loop:
| status | base | this PR |
|---|---|---|
| 200 | success | success |
| 404 | success | failure ✓ |
| 503 | failure | failure ✓ |
| 429 | failure | success (loses Retry-After backoff) |
| 500 / 502 | failure | success (falls into parse_results, r.json() raises) |
Since the intent is just "don't treat 404 as no-data, let transient errors retry," the minimal fix is:
def _api_response_is_success(self, r):
# crt.sh returns 404/503 transiently; don't short-circuit 404 as "no data"
return getattr(r, "is_success", False)That sends 404/503/500/502/429 all through the existing api_request retry/backoff path.
Happy to push this fix up if you'd rather hand it off, otherwise feel free to update the PR yourself and we can get it merged.
I've been observing anomalous API responses from the crt API returning 404 and 503 status codes, which may trigger especially if we abuse(?) the API by repeatedly querying the same domain from the same IP. This change shouldn't affect most domains and users, but should be an added safeguard in case the API goes wonky.