Adding Tavily Search API as a tool. (#600)

* Adding Tavily Search API as a tool.

* fix formatting + type List

llama_hub/tools/library.json

@@ -102,6 +102,10 @@
     "id": "tools/slack",
     "author": "jerryjliu"
   },
+  "TavilyToolSpec": {
+    "id": "tools/tavily",
+    "author": "rotemweiss57"
+  },
   "TextToImageToolSpec": {
     "id": "tools/text_to_image",
     "author": "ajhofmann"

llama_hub/tools/notebooks/tavily.ipynb

@@ -0,0 +1,195 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "bd2b2eba-b7fd-4856-960f-f2cbadcc12af",
+   "metadata": {},
+   "source": [
+    "# Building a Tavily Data Agent\n",
+    "\n",
+    "This tutorial walks through using the LLM tools provided by the [Tavily API](https://app.tavily.com/) to allow LLMs to easily search and retrieve relevant content from the Internet.\n",
+    "\n",
+    "To get started, you will need an [OpenAI api key](https://platform.openai.com/account/api-keys) and a [Tavily API key](https://app.tavily.com)\n",
+    "\n",
+    "We will import the relevant agents and tools and pass them our keys here:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "df2a0ecd-22e9-4cef-b069-89e4286e4d75",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-10-25T09:55:23.881158Z",
+     "start_time": "2023-10-25T09:55:23.865840Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "search\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Set up OpenAI\n",
+    "import openai\n",
+    "from llama_index.agent import OpenAIAgent\n",
+    "\n",
+    "openai.api_key = \"sk-api-key\"\n",
+    "\n",
+    "# Set up Tavily tool\n",
+    "from llama_hub.tools.tavily.base import TavilyToolSpec\n",
+    "\n",
+    "tavily_tool = TavilyToolSpec(\n",
+    "    api_key=\"tvly-api-key\",\n",
+    ")\n",
+    "\n",
+    "tavily_tool_list = tavily_tool.to_tool_list()\n",
+    "for tool in tavily_tool_list:\n",
+    "    print(tool.metadata.name)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "fe8e3012-bab0-4e55-858a-e3721282552c",
+   "metadata": {},
+   "source": [
+    "## Testing the Tavily search tool\n",
+    "\n",
+    "We've imported our OpenAI agent, set up the api key, and initialized our tool. Let's test out the tool before setting up our Agent.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "e64da618-b4ab-42d7-903d-f4eeb624f43c",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-10-25T09:55:40.329249Z",
+     "start_time": "2023-10-25T09:55:25.783993Z"
+    }
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[Document(id_='8038fd7d-10dc-4d7a-b535-51b2b56f7c3e', embedding=None, metadata={'url': 'https://www.npr.org/2023/09/05/1197587557/burning-man-exodus-2023-mud'}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, hash='be58b5ad268c8d25cd4961eaa603e4ec5d0a56ce994222bb8a1315e1d572e5d4', text=',\\nChloe Veltman\\nVehicles seen departing the Burning Man festival in Black Rock City, Nev., on Monday.\\nMatt Mills McKnight/Reuters\\nhide caption\\n\\nNational\\nThe latest on the Burning Man flooding\\nSome frustration, however, started to seep in for some attendees by Sunday.\\n\\nCulture\\nBurning Man festival organizers lift driving ban and allow people to leave\\n\\nhide caption\\nVehicles seen departing the Burning Man festival in Black Rock City, Nev., on Monday.\\n\\nEvent volunteer Josh Lease said that in true Burning Man spirit, people were sharing warm clothes and phone chargers where they could — and they kept the music blaring.', start_char_idx=None, end_char_idx=None, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'),\n",
+       " Document(id_='792a9e12-8de7-40de-9710-824b04e6e10c', embedding=None, metadata={'url': 'https://www.insider.com/explainer-everything-that-went-wrong-burning-man-festival-so-far-2023-9'}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, hash='246ed39cfaca5d8f5a5a365d7779de0e3b9b6fa7594b4b9041e39bac11174bbd', text='And now,\\xa0tens of thousands of Burning Man attendees are stranded due to a rainstorm that turned the desert into a mud pit.\\nHere\\'s how Burning Man 2023 has played out so far.\\n\\nJump to\\nEverything that went wrong at Burning Man 2023, so far\\nThe year\\'s Burning Man is off to a rocky start.\\n\\nA Burning Man festival organizer told Insider in a statement that organizers conducted \"table-top drills for events like this\" and are working to provide an exit for stranded attendees.\\n\\nto an X post from Burning Man Traffic, a social media account that shares updates about the campsite.\\n\\nThe Guardian reported that four protesters were arrested, and local authorities removed the blockade.\\nBurning Man advised guests to shelter in place and preserve resources as bad weather continued.', start_char_idx=None, end_char_idx=None, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n'),\n",
+       " Document(id_='e2868174-2ea2-444a-8571-28ce840ef47f', embedding=None, metadata={'url': 'https://www.billboard.com/culture/events/burning-man-2023-disaster-recap-1235407933/'}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, hash='97de1e53e980fbf8b56772a6d76a5f89f283bec7954d6b049c370fc164ccde54', text=\"At around 4 a.m. last Saturday an unusual thing happened at Burning Man. The event, typically an all-hours hubbub of music, art cars, laughter, weeping, whirring bikes, bass drops and other assorted\\n\\nBurning Man 2023: Relatively Normal, Honestly\\xa0Great\\nThe 2023 gathering in Nevada's Black Rock Desert was called a disaster, but that didn't feel true for those on the inside of it.\\nBy\\nKatie Bain\\n\\nthe next few days of our collective existence became a question mark. But also, uncertainty is exciting, and why go to Burning Man but for an unconventional experience? There was a buzz in the air\\n\\nany type of vehicle without getting stuck or, worse, slice ruts in the roads making driving difficult once the ground dried. I’ve been to Burning Man eight times\\xa0— for fun, for adventure, for work.\\n\\nthe flooded tents and Burners with plastic grocery bags duct taped around their shoes, one couldn’t help but laugh. Or cry. Or both.\", start_char_idx=None, end_char_idx=None, text_template='{metadata_str}\\n\\n{content}', metadata_template='{key}: {value}', metadata_seperator='\\n')]"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "tavily_tool.search(\"What happened in the latest Burning Man festival?\", max_results=3)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1210906d-87a7-466a-9712-1d17dba2c2ec",
+   "metadata": {},
+   "source": [
+    "### Using the Search tool in an Agent\n",
+    "\n",
+    "We can create an agent with access to the Tavily search tool start testing it out:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "9d88c2ee-184a-4371-995b-a086b34db24f",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-10-25T09:55:48.339621Z",
+     "start_time": "2023-10-25T09:55:48.332915Z"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "agent = OpenAIAgent.from_tools(\n",
+    "    tavily_tool_list,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "f69a53fd-55c4-4e18-8fbe-6a29d5f3cef0",
+   "metadata": {
+    "ExecuteTime": {
+     "end_time": "2023-10-25T09:55:57.109392Z",
+     "start_time": "2023-10-25T09:55:50.456904Z"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "# Analysis of the Latest Burning Man Floods\n",
+      "\n",
+      "## Introduction\n",
+      "The Burning Man festival, held annually in the Black Rock Desert of Nevada, is known for its unique art installations, music performances, and community spirit. However, in recent years, the festival has faced challenges due to extreme weather conditions, including floods. This analysis aims to provide a deep understanding of the latest Burning Man floods, their causes, impacts, and potential solutions.\n",
+      "\n",
+      "## Causes of the Floods\n",
+      "1. Heavy Rainfall: The primary cause of the floods at Burning Man is heavy rainfall. The desert region where the festival takes place is typically arid, but occasional rainstorms can result in significant amounts of water accumulating on the playa.\n",
+      "2. Poor Drainage System: The infrastructure at Burning Man is not designed to handle large amounts of rainfall. The lack of an efficient drainage system exacerbates the flooding problem, as the water has no proper outlet to flow away from the festival grounds.\n",
+      "3. Climate Change: The increasing frequency and intensity of extreme weather events, including heavy rainfall, can be attributed to climate change. As global temperatures rise, the likelihood of extreme weather events occurring during the festival also increases.\n",
+      "\n",
+      "## Impacts of the Floods\n",
+      "1. Damage to Art Installations: The floods can cause significant damage to the intricate and often fragile art installations at Burning Man. The water can erode structures, ruin electronic components, and render artworks unsalvageable.\n",
+      "2. Safety Concerns: The floods pose safety risks to festival attendees. Standing water can create slippery conditions, increasing the likelihood of accidents and injuries. Additionally, flash floods can occur suddenly, catching people off guard and potentially leading to dangerous situations.\n",
+      "3. Disruption of Activities: The floods can disrupt scheduled performances, workshops, and other activities at Burning Man. The festival relies on a carefully planned itinerary, and any interruptions due to flooding can result in disappointment and logistical challenges.\n",
+      "\n",
+      "## Response and Mitigation Measures\n",
+      "1. Improved Drainage System: One of the key solutions to address the flooding issue is the implementation of a more robust and efficient drainage system. This would involve creating channels or trenches to direct water away from the festival grounds, preventing it from accumulating and causing floods.\n",
+      "2. Weather Monitoring and Early Warning Systems: Installing weather monitoring equipment and establishing early warning systems can help organizers anticipate and prepare for potential floods. This would allow them to take proactive measures, such as evacuating certain areas or rescheduling activities, to ensure the safety of attendees.\n",
+      "3. Artwork Protection Measures: Artists and organizers can take steps to protect art installations from flood damage. This may include using waterproof materials, elevating structures, or designing installations that can withstand water exposure.\n",
+      "4. Climate Change Adaptation: Given the role of climate change in exacerbating extreme weather events, it is crucial for Burning Man organizers to consider long-term adaptation strategies. This may involve exploring alternative locations less prone to flooding or implementing sustainable practices to mitigate the festival's carbon footprint.\n",
+      "\n",
+      "## Conclusion\n",
+      "The latest Burning Man floods highlight the vulnerability of the festival to extreme weather events. Addressing the causes and impacts of these floods requires a multi-faceted approach, including improved infrastructure, enhanced monitoring systems, and climate change adaptation measures. By implementing these solutions, Burning Man can continue to thrive as a unique and vibrant event while ensuring the safety and preservation of its art and community.\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(agent.chat(\"Write a deep analysis in markdown syntax about the latest burning man floods\"))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "555c6c1a",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

llama_hub/tools/tavily/README.md

@@ -0,0 +1,28 @@
+# Tavily Tool
+
+[Tavily](https://app.tavily.com/) is a robust search API tailored specifically for LLM Agents. It seamlessly integrates with diverse data sources to ensure a superior, relevant search experience.
+
+To begin, you need to obtain an API key on the [Tavily's developer dashboard](https://app.tavily.com/).
+
+## Usage
+
+This tool has a more extensive example usage documented in a Jupyter notebook [here](https://github.com/emptycrown/llama-hub/tree/main/llama_hub/tools/notebooks/tavily.ipynb)
+
+Here's an example usage of the TavilyToolSpec.
+
+```python
+from llama_hub.tools.tavily import TavilyToolSpec
+from llama_index.agent import OpenAIAgent
+
+tavily_tool = TavilyToolSpec(
+    api_key='your-key',
+)
+agent = OpenAIAgent.from_tools(tavily_tool.to_tool_list())
+
+agent.chat('What happened in the latest Burning Man festival?')
+```
+
+`search`: Search for relevant dynamic data based on a query. Returns a list of urls and their relevant content.
+
+
+This loader is designed to be used as a way to load data as a Tool in an Agent. See [here](https://github.com/emptycrown/llama-hub/tree/main) for examples.

llama_hub/tools/tavily/__init__.py

@@ -0,0 +1,6 @@
+# init
+from llama_hub.tools.tavily.base import (
+    TavilyToolSpec,
+)
+
+__all__ = ["TavilyToolSpec"]

llama_hub/tools/tavily/base.py

@@ -0,0 +1,41 @@
+"""Tavily tool spec."""
+
+from typing import Optional, List
+from llama_index.readers.schema.base import Document
+from llama_index.tools.tool_spec.base import BaseToolSpec
+
+
+class TavilyToolSpec(BaseToolSpec):
+    """Tavily tool spec."""
+
+    spec_functions = [
+        "search",
+    ]
+
+    def __init__(self, api_key: str) -> None:
+        """Initialize with parameters."""
+        from tavily import Client
+
+        self.client = Client(api_key=api_key)
+
+    def search(self, query: str, max_results: Optional[int] = 6) -> List[Document]:
+        """
+        Run query through Tavily Search and return metadata.
+
+        Args:
+            query: The query to search for.
+            max_results: The maximum number of results to return.
+
+        Returns:
+            results: A list of dictionaries containing the results:
+                url: The url of the result.
+                content: The content of the result.
+
+        """
+        response = self.client.search(
+            query, max_results=max_results, search_depth="advanced"
+        )
+        return [
+            Document(text=result["content"], extra_info={"url": result["url"]})
+            for result in response["results"]
+        ]

llama_hub/tools/tavily/requirements.txt

@@ -0,0 +1 @@
+tavily-python