Skip to content

fix: guard against anomalous crt API responses#3077

Closed
TrebledJ wants to merge 1 commit into
blacklanternsecurity:devfrom
TrebledJ:patch-5
Closed

fix: guard against anomalous crt API responses#3077
TrebledJ wants to merge 1 commit into
blacklanternsecurity:devfrom
TrebledJ:patch-5

Conversation

@TrebledJ
Copy link
Copy Markdown
Contributor

@TrebledJ TrebledJ commented May 4, 2026

I've been observing anomalous API responses from the crt API returning 404 and 503 status codes, which may trigger especially if we abuse(?) the API by repeatedly querying the same domain from the same IP. This change shouldn't affect most domains and users, but should be an added safeguard in case the API goes wonky.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 8, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91%. Comparing base (5be4993) to head (0541deb).
⚠️ Report is 150 commits behind head on dev.

Additional details and impacted files
@@          Coverage Diff          @@
##             dev   #3077   +/-   ##
=====================================
+ Coverage     91%     91%   +1%     
=====================================
  Files        437     437           
  Lines      37509   37511    +2     
=====================================
+ Hits       33925   33927    +2     
  Misses      3584    3584           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@liquidsec liquidsec self-assigned this May 12, 2026
Copy link
Copy Markdown
Collaborator

@liquidsec liquidsec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this. The 404/503 handling is definitely a real gap. However as written this short-circuits the existing api retry logic:

return getattr(r, "is_success", False) or getattr(r, "status_code", 0) not in {404, 503}

The second clause is True for any status code other than 404/503, so 429, 500, 502, etc. all get marked as success and skip the retry loop:

status base this PR
200 success success
404 success failure ✓
503 failure failure ✓
429 failure success (loses Retry-After backoff)
500 / 502 failure success (falls into parse_results, r.json() raises)

Since the intent is just "don't treat 404 as no-data, let transient errors retry," the minimal fix is:

def _api_response_is_success(self, r):
    # crt.sh returns 404/503 transiently; don't short-circuit 404 as "no data"
    return getattr(r, "is_success", False)

That sends 404/503/500/502/429 all through the existing api_request retry/backoff path.

Happy to push this fix up if you'd rather hand it off, otherwise feel free to update the PR yourself and we can get it merged.

@liquidsec liquidsec added the waiting Waiting on upstream / third party label May 21, 2026
@liquidsec
Copy link
Copy Markdown
Collaborator

@TrebledJ we ended up needing to do a big more with crt and also ended up needing it more urgently, so we've moved this to #3124

@liquidsec liquidsec closed this May 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting Waiting on upstream / third party

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants