Skip to content

feat(node): add tool_tavily web-search agent tool - #1097

Merged
asclearuc merged 9 commits into
rocketride-org:developfrom
EdwardLien0426:feat/RR-1047-tool-tavily
Jun 4, 2026
Merged

asclearuc merged 9 commits into
rocketride-org:developfrom
EdwardLien0426:feat/RR-1047-tool-tavily

Conversation

@EdwardLien0426

@EdwardLien0426 EdwardLien0426 commented Jun 3, 2026 •

Copy link
Copy Markdown
Contributor

Brings the Tavily web-search tool (tool_tavily) into develop.

The same change was merged to stage as #1072 to make the Nebius event cut; this rebases the branch onto develop for the long-term line. Same content (including the keyless-CI dynamic-test fix and the shared ai.common.utils helpers — post_with_retry, validate_public_url), no conflicts.

Part of #1047.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added Tavily web-search tool for agents with configurable API key, max results, search depth, and topic.
    • Improved network robustness and URL safety via shared HTTP retry and public-URL validation utilities.
  • Documentation

    • Added "Agent Tools" subsection to pipeline node docs with Tavily details and setup instructions.
  • Tests

    • Added unit tests for result formatting, HTTP retry behavior, and URL validation.

EdwardLien0426 and others added 8 commits June 3, 2026 13:44
Adds tool_tavily, exposing Tavily real-time web search as an agent tool
(POST https://api.tavily.com/search) via the @tool_function "tavily".
classType ["tool"], lanes {}, invoked through an agent's control-plane
tool channel. Includes retry with backoff on 429/5xx, an SSRF guard on
returned URLs, TAVILY_API_KEY env fallback, and unit tests for the
result-shaping and URL-validation helpers. Cloned from tool_exa_search.

Pairs with llm_nebius + agent_deepagent for agentic web search.

Fixes rocketride-org#1047

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- tavily.svg: replace the generic magnifying-glass placeholder with the
  official Tavily brand mark (icon portion of the official logo).
- IGlobal: strip the configured apikey before falling back to
  TAVILY_API_KEY, so a whitespace-only config value no longer shadows
  the env var (and the "key required" error stays accurate).
- IInstance: retry requests.exceptions.ConnectionError alongside Timeout
  (transient transport failures were previously treated as fatal).
- Add a unit test for the ConnectionError retry path.
- docs: clarify tool_tavily is agent-agnostic (attaches to any agent's
  tool channel), not coupled to agent_deepagent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- IGlobal: wrap int(maxResults) in try/except so a non-numeric config
  value falls back to the default 5 with a clear message instead of
  crashing beginGlobal with a raw ValueError.
- README: add a language specifier (bash) to the env-var code fence
  (markdownlint MD040).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The import stub mocked `requests` but only set Timeout and
exceptions.RequestException as real classes, leaving
exceptions.ConnectionError (and the top-level requests.RequestException
that IInstance references) as MagicMocks. Once the retry helper began
catching ConnectionError, the except clause raised
"catching classes that do not inherit from BaseException" under the
stub, failing the new retry test in CI. Point both at real exception
classes. Verified: 4 passed via `python3 -m pytest`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
resp.json() could raise ValueError on a 200 with a non-JSON body, which
tavily() does not catch (it only handles RuntimeError) — so it escaped
the {'success': False, ...} error contract and crashed the tool call. A
valid-but-non-object payload (list/string) also broke _shape_results'
body.get(...). Decode defensively in _request_with_retry: convert bad
JSON to RuntimeError and reject non-dict payloads. Adds a unit test for
the non-dict case. Verified: 5 passed via python3 -m pytest.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…util

Per @asclearuc / @dsapandora review on rocketride-org#1072:

- Rename the API-key env var TAVILY_API_KEY -> ROCKETRIDE_TAVILY_KEY
  (RocketRide only substitutes ROCKETRIDE_-prefixed vars, and the
  node-test framework maps ROCKETRIDE_<PROVIDER>_<ATTR>). Defined as a
  module constant instead of a magic string. Updated IGlobal, services
  test `requires`, and README.
- Move the SSRF URL guard out of the node into the shared
  ai.common.utils.validate_public_url (deduped from the search_exa copy;
  search_exa untouched, to be switched over in the dedup PR). Added
  ai-package tests for it; the tool's test stubs it as a pass-through.
- time_range enum now uses sorted(VALID_TIME_RANGES) for consistency
  with search_depth / topic.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… helper

Per @asclearuc review on rocketride-org#1072: don't hand-roll a retry loop — use
tenacity (the repo's standard retry mechanism) and put the shared helper
in ai.common.utils.

- Add ai.common.utils.post_with_retry: a tenacity-based POST that retries
  timeouts, connection errors, and 429/5xx (exponential backoff), raising
  on a final non-429 4xx without retry. Added ai-package tests.
- tool_tavily IInstance now calls post_with_retry and does its own JSON /
  dict-shape validation; the local _request_with_retry loop (and the
  time/debug imports) are removed. search_exa / xtrace_memory untouched —
  they switch over in the dedup PR.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Register tool_tavily in _LLM_MOCK_CREDENTIALS and drop the
ROCKETRIDE_TAVILY_KEY `requires` gate so the dynamic test builds the
node under ROCKETRIDE_MOCK with a placeholder key instead of being
skipped when no key is present (matches tool_exa_search).
@coderabbitai

coderabbitai Bot commented Jun 3, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 6d199cbc-d5e2-4414-a0d9-d4a074bc6e38

📥 Commits

Reviewing files that changed from the base of the PR and between 36352ad and e91b320.

📒 Files selected for processing (5)
  • nodes/src/nodes/tool_tavily/IInstance.py
  • nodes/test/test_tool_tavily.py
  • packages/ai/src/ai/common/utils/url_utils.py
  • packages/ai/tests/ai/common/utils/test_http_retry.py
  • packages/ai/tests/ai/common/utils/test_url_utils.py

📝 Walkthrough

Walkthrough

Adds a Tavily web-search agent tool node with IGlobal state and IInstance tooling, shared post_with_retry and SSRF-safe validate_public_url utilities, service metadata and README, plus unit tests and test-framework mock credentials.

Changes

Tavily Tool Node and Infrastructure

Layer / File(s) Summary
Shared HTTP retry and URL validation utilities
packages/ai/src/ai/common/utils/http_retry.py, packages/ai/src/ai/common/utils/url_utils.py, packages/ai/src/ai/common/utils/__init__.py
New post_with_retry implements tenacity-backed POST retries on timeouts, connection errors, and 429/5xx (not other 4xx). validate_public_url enforces http(s), resolves hostnames, and rejects non-global IP resolutions to mitigate SSRF; both are re-exported in ai.common.utils.
Tavily global state, package, and service config
nodes/src/nodes/tool_tavily/IGlobal.py, nodes/src/nodes/tool_tavily/services.json, nodes/src/nodes/tool_tavily/__init__.py, nodes/src/nodes/tool_tavily/requirements.txt
IGlobal loads the API key from node config or ROCKETRIDE_TAVILY_KEY, validates presence on beginGlobal, clamps maxResults (1–20), and normalizes searchDepth/topic. Service metadata declares secure apikey field, defaults, and UI constraints; package exports IGlobal/IInstance; requests added to requirements.
Tavily tool instance and API integration
nodes/src/nodes/tool_tavily/IInstance.py
IInstance.tavily validates inputs (non-empty query), applies defaults from IGlobal, builds request payload (time_range, include/exclude domains), calls Tavily via post_with_retry with Bearer auth, handles errors, and uses _shape_results (with validate_public_url) to return sanitized results.
Tool node documentation
nodes/src/nodes/tool_tavily/README.md, docs/README-nodes.md
New README documents tool_tavily usage and config (apikey via node config or ROCKETRIDE_TAVILY_KEY); main nodes docs add an "Agent Tools" subsection listing tool_tavily.
HTTP retry utility tests
packages/ai/tests/ai/common/utils/test_http_retry.py
Unit tests for post_with_retry: immediate 200 success, retries on Timeout/429/5xx, non-retry on non-429 4xx (raises), and retry exhaustion behavior.
URL validation utility tests
packages/ai/tests/ai/common/utils/test_url_utils.py
SSRF-focused tests: allow public HTTPS, reject non-http schemes, loopback literals, private/CGNAT ranges, and unresolvable hosts via DNS stubbing.
Tavily tool unit tests
nodes/test/test_tool_tavily.py
Pure-unit tests for _shape_results using lightweight import-time stubs; verifies field mapping and that non-dict upstream results are skipped.
Test framework mock credentials
nodes/test/framework/pipeline.py
Adds tool_tavily entry with a mock apikey placeholder used in the testing credential injection path under ROCKETRIDE_MOCK.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested reviewers

  • jmaionchi
  • stepmikhaylov
  • Rod-Christensen
  • asclearuc

🐰 A web-search tool hops into place,
Keys and retries snug in a secure embrace,
Results are shaped and unsafe links are barred,
Agents now fetch the web — swift, tidy, and hard.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 17.14% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding a tool_tavily web-search agent tool node to the codebase.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions github-actions Bot added docs Documentation module:nodes Python pipeline nodes module:ai AI/ML modules labels Jun 3, 2026
@github-actions

github-actions Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor
🤖 Internal: Discord sync marker

Auto-managed by the Discord notification workflow. Stores the linked Discord message ID. Do not edit or delete.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@nodes/src/nodes/tool_tavily/IInstance.py`:
- Around line 194-214: _shaped_results currently assumes each entry in
body.get('results') is a dict and will raise AttributeError for scalar or string
entries; update the _shape_results function to skip any non-dict items (e.g.,
add an isinstance(item, dict) guard before using item.get(...)) so malformed
upstream payloads are ignored rather than crashing, optionally track or log the
count of skipped items; keep existing validate_public_url handling for URLs and
preserve the current returned schema from _shape_results.

In `@packages/ai/src/ai/common/utils/url_utils.py`:
- Around line 24-34: The SSRF gate in validate_public_url currently checks a set
of individual ip properties (is_private, is_loopback, etc.) which misses CGNAT
addresses like 100.64.0.0/10; update the loop that iterates over addrinfo and
validates each sockaddr's IP (the ip variable) to reject any IP where
ip.is_global is False (i.e., if not ip.is_global: raise ValueError(...)),
replacing the existing combined property checks so any
non-public/reserved/non-global addresses are blocked; keep the same error
message using parsed.hostname to preserve context.

In `@packages/ai/tests/ai/common/utils/test_http_retry.py`:
- Around line 64-100: Add a new unit test that verifies post_with_retry will
retry on 5xx responses (e.g., first return a _FakeResp(500) then a
_FakeResp(200) and assert the successful response and call count), mirroring the
existing test_retries_429_then_succeeds pattern; reference the _is_retryable()
branch for 500-599 and the post_with_retry function to locate the relevant logic
and use _patch_post to monkeypatch requests as in the other tests.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: bc55c35c-54cd-45fd-b1f1-534b815e6f46

📥 Commits

Reviewing files that changed from the base of the PR and between 6886e43 and 36352ad.

⛔ Files ignored due to path filters (1)
  • nodes/src/nodes/tool_tavily/tavily.svg is excluded by !**/*.svg
📒 Files selected for processing (14)
  • docs/README-nodes.md
  • nodes/src/nodes/tool_tavily/IGlobal.py
  • nodes/src/nodes/tool_tavily/IInstance.py
  • nodes/src/nodes/tool_tavily/README.md
  • nodes/src/nodes/tool_tavily/__init__.py
  • nodes/src/nodes/tool_tavily/requirements.txt
  • nodes/src/nodes/tool_tavily/services.json
  • nodes/test/framework/pipeline.py
  • nodes/test/test_tool_tavily.py
  • packages/ai/src/ai/common/utils/__init__.py
  • packages/ai/src/ai/common/utils/http_retry.py
  • packages/ai/src/ai/common/utils/url_utils.py
  • packages/ai/tests/ai/common/utils/test_http_retry.py
  • packages/ai/tests/ai/common/utils/test_url_utils.py

Comment thread nodes/src/nodes/tool_tavily/IInstance.py
Comment thread packages/ai/src/ai/common/utils/url_utils.py
Comment thread packages/ai/tests/ai/common/utils/test_http_retry.py
…, 5xx test

- _shape_results: skip non-dict result items instead of crashing
- validate_public_url: use ip.is_global (also blocks CGNAT 100.64.0.0/10)
- add 5xx-retry test and CGNAT/non-dict regression tests

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@asclearuc
asclearuc merged commit ccc5d22 into rocketride-org:develop Jun 4, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation module:ai AI/ML modules module:nodes Python pipeline nodes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants