Skip to content

feat(tools): NVD rate-limit resilience — retry/back-off + NVD_API_KEY support#91

Merged
manus-use merged 1 commit into
manus-use:mainfrom
manusjs:feat/nvd-retry-backoff
Jul 5, 2026
Merged

feat(tools): NVD rate-limit resilience — retry/back-off + NVD_API_KEY support#91
manus-use merged 1 commit into
manus-use:mainfrom
manusjs:feat/nvd-retry-backoff

Conversation

@manusjs

@manusjs manusjs commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

NVD's public API enforces a rate limit of 5 requests per 30 seconds for
unauthenticated callers. Before this PR a single 429 or transient server
error was fatal — the tool returned an error immediately with no retry.

This PR adds production-grade resilience to get_nvd_data (the most-called
tool in every manus-agent analyze pipeline) and to obtain_cves (which
also paginating-polls the same NVD endpoint):


Changes

src/manus_agent/tools/get_nvd_data.py

What Detail
_build_nvd_headers() Injects NVD_API_KEY env var as apiKey header when set; returns empty dict otherwise. Key is never logged.
_nvd_get_with_retry(url) Performs up to NVD_MAX_RETRIES (default 3) retries with exponential back-off (NVD_RETRY_BASE_DELAY * 2^n, default 2 s / 4 s / 8 s) on HTTP 429, 500, 502, 503, 504. Non-retryable 4xx errors fail immediately. Connection/timeout errors are also retried. All constants are env-var-overridable for testing (set NVD_RETRY_BASE_DELAY=0).
get_nvd_data() Uses _nvd_get_with_retry() instead of bare requests.get().

src/manus_agent/tools/obtain_cves.py

_get_all_cves_from_nvd() (the pagination loop that calls the same NVD
endpoint) now imports and uses _nvd_get_with_retry for the same resilience
benefit, replacing the bare requests.get + raise_for_status() combo.

tests/test_nvd_retry.py — 37 new tests (902 → 939)

Class Tests Coverage
TestBuildNvdHeaders 3 no-key → empty, key → apiKey header, blank/whitespace not injected
TestNvdGetWithRetry 15 success, API-key forwarded, no-key empty headers, 429/500/503 retry-then-succeed, all-3-retries-then-succeed, exhausted-429-raises, connection-error retry, timeout retry, exhausted-connection-raises, back-off schedule, no sleep on attempt-1, 404-not-retried, 400-not-retried
TestGetNvdDataTool 4 success, 429-retry-then-success, exhausted-retries-error, invalid-cve-no-retry
TestObtainCvesNvdRetry 3 429-retry-in-pagination, single-page-success, exhausted-raises
TestRetryableStatusSet 6 429/500 in set, 200/404 not in set, parametrised all 5 expected codes
TestModuleConstants 4 default 3 retries, default 2s delay, obtain_cves uses same helper

All tests 100% mocked — no real HTTP calls.


Back-off schedule

attempt 1 → immediate
attempt 2 → sleep 2 s  (NVD_RETRY_BASE_DELAY × 2^0)
attempt 3 → sleep 4 s  (NVD_RETRY_BASE_DELAY × 2^1)
attempt 4 → sleep 8 s  (NVD_RETRY_BASE_DELAY × 2^2)

Optional API key

export NVD_API_KEY="your-key-here"
manus-agent analyze CVE-2024-3094   # now sends apiKey header, 50 req/30s limit

Test run

939 passed, 3 deselected, 0 failures

Open PRs checked for overlap (none duplicate this work)

#51, #53, #54, #58, #60, #64, #65, #67, #74, #75, #76, #77, #78, #79,
#80, #82, #83, #85, #86, #87, #88, #89, #90 — none cover NVD retry/backoff
or NVD_API_KEY support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants