Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
da47e0e
docs: add TavilyWebSearch component page and external integration entry
SyedShahmeerAli12 Mar 27, 2026
846de60
Merge branch 'main' into feat/add-tavily-docs
anakin87 Mar 27, 2026
5987a27
fix: `CountDocumentsAsyncTest`, `WriteDocumentsAsyncTest`, `WriteDocu…
pandego Mar 27, 2026
2ad51ae
docs: remove gpt-3.5-turbo mentions and use ChatMessage.txt (no conte…
anakin87 Mar 27, 2026
dc48f84
fix: `DeleteAllAsyncTest`, `DeleteByFilterAsyncTest`, (#10952)
pandego Mar 27, 2026
f97867d
Sync Haystack API reference on Docusaurus (#10959)
HaystackBot Mar 27, 2026
5052c96
fix: `UpdateByFilterAsyncTest`, `CountDocumentsByFilterAsyncTest`, `C…
pandego Mar 27, 2026
3de3a53
Sync Haystack API reference on Docusaurus (#10961)
HaystackBot Mar 27, 2026
b6d9f63
docs: fixing code snippets syntax errors (#10955)
davidsbatista Mar 27, 2026
92a3742
feat: add get_meta_data async mixin tests to haystack.testing + InMem…
davidsbatista Mar 27, 2026
f6ba812
Sync Haystack API reference on Docusaurus (#10962)
HaystackBot Mar 27, 2026
67b8035
docs: update llama.cpp repo links from ggerganov to ggml-org (#10964)
satishkc7 Mar 30, 2026
8368039
Sync Core Integrations API reference (nvidia) on Docusaurus (#10974)
HaystackBot Mar 30, 2026
36095ef
build: switch to trusted publishing (#10976)
julian-risch Mar 30, 2026
276d93e
test: adding mixing filter async tests + implementing them in InMemor…
davidsbatista Mar 30, 2026
7081e07
docs: address reviewer feedback on TavilyWebSearch docs
SyedShahmeerAli12 Mar 30, 2026
25254b3
Merge branch 'main' into feat/add-tavily-docs
anakin87 Mar 30, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs-website/docs/pipeline-components/websearch.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,4 @@ Use these components to look up answers on the internet.
| [FirecrawlWebSearch](websearch/firecrawlwebsearch.mdx) | Search engine using the Firecrawl API. |
| [SearchApiWebSearch](websearch/searchapiwebsearch.mdx) | Search engine using Search API. |
| [SerperDevWebSearch](websearch/serperdevwebsearch.mdx) | Search engine using SerperDev API. |
| [TavilyWebSearch](websearch/tavilywebsearch.mdx) | Search engine using the Tavily AI-powered search API. |
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@ External integrations that enable websearch with Haystack.
| --- | --- |
| [DuckDuckGo](https://haystack.deepset.ai/integrations/duckduckgo-api-websearch) | Use DuckDuckGo API for web searches. |
| [Exa](https://haystack.deepset.ai/integrations/exa) | Search the web with Exa's AI-powered search, get content, answers, and conduct deep research. |
| [Serpex](https://haystack.deepset.ai/integrations/serpex) | Multi-engine web search for Haystack — access Google, Bing, DuckDuckGo, Brave, Yahoo, and Yandex via Serpex API. |
| [Serpex](https://haystack.deepset.ai/integrations/serpex) | Multi-engine web search for Haystack — access Google, Bing, DuckDuckGo, Brave, Yahoo, and Yandex via Serpex API. |
103 changes: 103 additions & 0 deletions docs-website/docs/pipeline-components/websearch/tavilywebsearch.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
---
title: "TavilyWebSearch"
id: tavilywebsearch
slug: "/tavilywebsearch"
description: "Search engine using the Tavily AI-powered search API."
---

# TavilyWebSearch

Search the web using the Tavily AI-powered search API, optimized for LLM applications.

<div className="key-value-table">

| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) or right at the beginning of an indexing pipeline |
| **Mandatory init variables** | `api_key`: The Tavily API key. Can be set with the `TAVILY_API_KEY` env var. |
| **Mandatory run variables** | `query`: A string with your search query. |
| **Output variables** | `documents`: A list of Haystack Documents containing search result content and metadata. <br /> <br />`links`: A list of strings of resulting URLs. |
| **API reference** | [Tavily Search API](/reference/integrations-tavily) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/tavily/src/haystack_integrations/components/websearch/tavily/tavily_websearch.py |

</div>

## Overview

When you give `TavilyWebSearch` a query, it uses the [Tavily](https://tavily.com) Search API to search the web and return relevant content as Haystack `Document` objects. It also returns a list of the source URLs.

Tavily is an AI-powered search API built specifically for LLM applications. It returns clean, relevant snippets without the noise of traditional search engines, making it a great fit for RAG pipelines.

`TavilyWebSearch` requires a Tavily API key to work. By default, it looks for a `TAVILY_API_KEY` environment variable. Alternatively, you can pass an `api_key` directly during initialization.

## Usage

### On its own

Here is a quick example of how `TavilyWebSearch` searches the web based on a query and returns a list of Documents.

```python
from haystack_integrations.components.websearch.tavily import TavilyWebSearch
from haystack.utils import Secret

web_search = TavilyWebSearch(
api_key=Secret.from_env_var("TAVILY_API_KEY"),
top_k=5,
)
query = "What is Haystack by deepset?"

response = web_search.run(query=query)

for doc in response["documents"]:
print(doc.content)
```

### In a pipeline

Here is an example of a Retrieval-Augmented Generation (RAG) pipeline that uses `TavilyWebSearch` to look up an answer on the web.

```python
from haystack import Pipeline
from haystack.utils import Secret
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack_integrations.components.websearch.tavily import TavilyWebSearch
from haystack.dataclasses import ChatMessage

web_search = TavilyWebSearch(
api_key=Secret.from_env_var("TAVILY_API_KEY"),
top_k=3,
)

prompt_template = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user(
"Given the information below:\n"
"{% for document in documents %}{{ document.content }}\n{% endfor %}\n"
"Answer the following question: {{ query }}.\nAnswer:",
),
]

prompt_builder = ChatPromptBuilder(
template=prompt_template,
required_variables={"query", "documents"},
)

llm = OpenAIChatGenerator(
api_key=Secret.from_env_var("OPENAI_API_KEY"),
)

pipe = Pipeline()
pipe.add_component("search", web_search)
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)

pipe.connect("search.documents", "prompt_builder.documents")
pipe.connect("prompt_builder.prompt", "llm.messages")

query = "What is Haystack by deepset?"

result = pipe.run(data={"search": {"query": query}, "prompt_builder": {"query": query}})

print(result["llm"]["replies"][0].text)
```
1 change: 1 addition & 0 deletions docs-website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -613,6 +613,7 @@ export default {
'pipeline-components/websearch/firecrawlwebsearch',
'pipeline-components/websearch/searchapiwebsearch',
'pipeline-components/websearch/serperdevwebsearch',
'pipeline-components/websearch/tavilywebsearch',
'pipeline-components/websearch/external-integrations-websearch',
],
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,4 @@ Use these components to look up answers on the internet.
| [FirecrawlWebSearch](websearch/firecrawlwebsearch.mdx) | Search engine using the Firecrawl API. |
| [SearchApiWebSearch](websearch/searchapiwebsearch.mdx) | Search engine using Search API. |
| [SerperDevWebSearch](websearch/serperdevwebsearch.mdx) | Search engine using SerperDev API. |
| [TavilyWebSearch](websearch/tavilywebsearch.mdx) | Search engine using the Tavily AI-powered search API. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
---
title: "TavilyWebSearch"
id: tavilywebsearch
slug: "/tavilywebsearch"
description: "Search engine using the Tavily AI-powered search API."
---

# TavilyWebSearch

Search the web using the Tavily AI-powered search API, optimized for LLM applications.

<div className="key-value-table">

| | |
| --- | --- |
| **Most common position in a pipeline** | Before a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) or right at the beginning of an indexing pipeline |
| **Mandatory init variables** | `api_key`: The Tavily API key. Can be set with the `TAVILY_API_KEY` env var. |
| **Mandatory run variables** | `query`: A string with your search query. |
| **Output variables** | `documents`: A list of Haystack Documents containing search result content and metadata. <br /> <br />`links`: A list of strings of resulting URLs. |
| **API reference** | [Tavily Search API](/reference/integrations-tavily) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/tavily/src/haystack_integrations/components/websearch/tavily/tavily_websearch.py |

</div>

## Overview

When you give `TavilyWebSearch` a query, it uses the [Tavily](https://tavily.com) Search API to search the web and return relevant content as Haystack `Document` objects. It also returns a list of the source URLs.

Tavily is an AI-powered search API built specifically for LLM applications. It returns clean, relevant snippets without the noise of traditional search engines, making it a great fit for RAG pipelines.

`TavilyWebSearch` requires a Tavily API key to work. By default, it looks for a `TAVILY_API_KEY` environment variable. Alternatively, you can pass an `api_key` directly during initialization.

## Usage

### On its own

Here is a quick example of how `TavilyWebSearch` searches the web based on a query and returns a list of Documents.

```python
from haystack_integrations.components.websearch.tavily import TavilyWebSearch
from haystack.utils import Secret

web_search = TavilyWebSearch(
api_key=Secret.from_env_var("TAVILY_API_KEY"),
top_k=5,
)
query = "What is Haystack by deepset?"

response = web_search.run(query=query)

for doc in response["documents"]:
print(doc.content)
```

### In a pipeline

Here is an example of a Retrieval-Augmented Generation (RAG) pipeline that uses `TavilyWebSearch` to look up an answer on the web.

```python
from haystack import Pipeline
from haystack.utils import Secret
from haystack.components.builders.chat_prompt_builder import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack_integrations.components.websearch.tavily import TavilyWebSearch
from haystack.dataclasses import ChatMessage

web_search = TavilyWebSearch(
api_key=Secret.from_env_var("TAVILY_API_KEY"),
top_k=3,
)

prompt_template = [
ChatMessage.from_system("You are a helpful assistant."),
ChatMessage.from_user(
"Given the information below:\n"
"{% for document in documents %}{{ document.content }}\n{% endfor %}\n"
"Answer the following question: {{ query }}.\nAnswer:",
),
]

prompt_builder = ChatPromptBuilder(
template=prompt_template,
required_variables={"query", "documents"},
)

llm = OpenAIChatGenerator(
api_key=Secret.from_env_var("OPENAI_API_KEY"),
)

pipe = Pipeline()
pipe.add_component("search", web_search)
pipe.add_component("prompt_builder", prompt_builder)
pipe.add_component("llm", llm)

pipe.connect("search.documents", "prompt_builder.documents")
pipe.connect("prompt_builder.prompt", "llm.messages")

query = "What is Haystack by deepset?"

result = pipe.run(data={"search": {"query": query}, "prompt_builder": {"query": query}})

print(result["llm"]["replies"][0].text)
```
Original file line number Diff line number Diff line change
Expand Up @@ -609,6 +609,7 @@
"pipeline-components/websearch/firecrawlwebsearch",
"pipeline-components/websearch/searchapiwebsearch",
"pipeline-components/websearch/serperdevwebsearch",
"pipeline-components/websearch/tavilywebsearch",
"pipeline-components/websearch/external-integrations-websearch"
]
},
Expand Down
Loading