Skip to content

Commit

Permalink
WIP: update structured outputs syntax (#14747)
Browse files Browse the repository at this point in the history
  • Loading branch information
@jerryjliu
jerryjliu authored Jul 23, 2024
1 parent cc0ea90 commit 1e361b5
Show file tree
Hide file tree
Showing 25 changed files with 1,852 additions and 385 deletions.
140 changes: 139 additions & 1 deletion docs/docs/examples/llm/anthropic.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -374,7 +374,7 @@
"id": "5152a2b4-78e6-47a5-933d-f5186ec0f775",
"metadata": {},
"source": [
"## Async"
"### Async"
]
},
{
Expand Down Expand Up @@ -419,6 +419,144 @@
"source": [
"print(resp)"
]
},
{
"cell_type": "markdown",
"id": "e33bfa1f-589e-475a-8eb3-fa37d95759a7",
"metadata": {},
"source": [
"## Structured Prediction\n",
"\n",
"LlamaIndex provides an intuitive interface for converting any Anthropic LLMs into a structured LLM through `structured_predict` - simply define the target Pydantic class (can be nested), and given a prompt, we extract out the desired object."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d8b622d1-0c91-4cde-ab4c-3f83e0127a4b",
"metadata": {},
"outputs": [],
"source": [
"from llama_index.llms.anthropic import Anthropic\n",
"from llama_index.core.prompts import PromptTemplate\n",
"from llama_index.core.bridge.pydantic import BaseModel\n",
"from typing import List\n",
"\n",
"\n",
"class MenuItem(BaseModel):\n",
" \"\"\"A menu item in a restaurant.\"\"\"\n",
"\n",
" course_name: str\n",
" is_vegetarian: bool\n",
"\n",
"\n",
"class Restaurant(BaseModel):\n",
" \"\"\"A restaurant with name, city, and cuisine.\"\"\"\n",
"\n",
" name: str\n",
" city: str\n",
" cuisine: str\n",
" menu_items: List[MenuItem]\n",
"\n",
"\n",
"llm = Anthropic(\"claude-3-5-sonnet-20240620\")\n",
"prompt_tmpl = PromptTemplate(\n",
" \"Generate a restaurant in a given city {city_name}\"\n",
")\n",
"\n",
"# Option 1: Use `as_structured_llm`\n",
"restaurant_obj = (\n",
" llm.as_structured_llm(Restaurant)\n",
" .complete(prompt_tmpl.format(city_name=\"Miami\"))\n",
" .raw\n",
")\n",
"# Option 2: Use `structured_predict`\n",
"# restaurant_obj = llm.structured_predict(Restaurant, prompt_tmpl, city_name=\"Miami\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ec769384-d3fe-4761-99e3-bdddcb9c7e4d",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Restaurant(name='Ocean Breeze Bistro', city='Miami', cuisine='Seafood', menu_items=[MenuItem(course_name='Grilled Mahi-Mahi', is_vegetarian=False), MenuItem(course_name='Coconut Shrimp', is_vegetarian=False), MenuItem(course_name='Key Lime Pie', is_vegetarian=True), MenuItem(course_name='Vegetable Paella', is_vegetarian=True)])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"restaurant_obj"
]
},
{
"cell_type": "markdown",
"id": "c63fb270-3d7d-4b9c-9cda-aaa42004c7dd",
"metadata": {},
"source": [
"#### Structured Prediction with Streaming\n",
"\n",
"Any LLM wrapped with `as_structured_llm` supports streaming through `stream_chat`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e6e91261-e4ef-4706-8325-65d782f9a87b",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'city': 'San Francisco',\n",
" 'cuisine': 'California Fusion',\n",
" 'menu_items': [{'course_name': 'Sourdough Avocado Toast',\n",
" 'is_vegetarian': True},\n",
" {'course_name': 'Dungeness Crab Cioppino',\n",
" 'is_vegetarian': False},\n",
" {'course_name': 'Mission-style Veggie Burrito',\n",
" 'is_vegetarian': True},\n",
" {'course_name': 'Grilled Napa Valley Lamb Chops',\n",
" 'is_vegetarian': False},\n",
" {'course_name': 'Vegan Ghirardelli Chocolate Mousse',\n",
" 'is_vegetarian': True}],\n",
" 'name': 'Golden Gate Grill'}\n"
]
},
{
"data": {
"text/plain": [
"Restaurant(name='Golden Gate Grill', city='San Francisco', cuisine='California Fusion', menu_items=[MenuItem(course_name='Sourdough Avocado Toast', is_vegetarian=True), MenuItem(course_name='Dungeness Crab Cioppino', is_vegetarian=False), MenuItem(course_name='Mission-style Veggie Burrito', is_vegetarian=True), MenuItem(course_name='Grilled Napa Valley Lamb Chops', is_vegetarian=False), MenuItem(course_name='Vegan Ghirardelli Chocolate Mousse', is_vegetarian=True)])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from llama_index.core.llms import ChatMessage\n",
"from IPython.display import clear_output\n",
"from pprint import pprint\n",
"\n",
"input_msg = ChatMessage.from_str(\"Generate a restaurant in San Francisco\")\n",
"\n",
"sllm = llm.as_structured_llm(Restaurant)\n",
"stream_output = sllm.stream_chat([input_msg])\n",
"for partial_output in stream_output:\n",
" clear_output(wait=True)\n",
" pprint(partial_output.raw.dict())\n",
" restaurant_obj = partial_output.raw\n",
"\n",
"restaurant_obj"
]
}
],
"metadata": {
Expand Down
69 changes: 63 additions & 6 deletions docs/docs/examples/llm/mistralai.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"Paul Graham is a well-known entrepreneur, hacker, and essayist. He co-founded the startup incubator Y Combinator in 2005, which has since become one of the most successful and influential startup accelerators in the world. Graham is also known for his essays on entrepreneurship, programming, and startups, which have been published on his website, Hacker News, and in various publications. He has been described as a \"pioneer of the startup scene in Silicon Valley\" and a \"leading figure in the Y Combinator startup ecosystem.\" Graham's essays have influenced generations of entrepreneurs and programmers, and he is widely regarded as a thought leader in the tech industry.\n"
"Paul Graham is a well-known American computer programmer, entrepreneur, and essayist. He was born on February 24, 1964. He co-founded the startup incubator Y Combinator in 2005, which has been instrumental in the success of many tech companies, including Airbnb, Dropbox, and Stripe. Graham is also known for his essays on startup culture, programming, and entrepreneurship, which have been widely read and influential.\n"
]
}
],
Expand Down Expand Up @@ -638,7 +638,7 @@
"source": [
"## Structured Prediction\n",
"\n",
"An important use case for function calling is extracting structured objects. LlamaIndex provides an intuitive interface for this through `structured_predict` - simply define the target Pydantic class (can be nested), and given a prompt, we extract out the desired object."
"An important use case for function calling is extracting structured objects. LlamaIndex provides an intuitive interface for converting any LLM into a structured LLM - simply define the target Pydantic class (can be nested), and given a prompt, we extract out the desired object."
]
},
{
Expand All @@ -650,7 +650,7 @@
"source": [
"from llama_index.llms.mistralai import MistralAI\n",
"from llama_index.core.prompts import PromptTemplate\n",
"from pydantic import BaseModel\n",
"from llama_index.core.bridge.pydantic import BaseModel\n",
"\n",
"\n",
"class Restaurant(BaseModel):\n",
Expand All @@ -665,9 +665,15 @@
"prompt_tmpl = PromptTemplate(\n",
" \"Generate a restaurant in a given city {city_name}\"\n",
")\n",
"restaurant_obj = llm.structured_predict(\n",
" Restaurant, prompt_tmpl, city_name=\"Miami\"\n",
")"
"\n",
"# Option 1: Use `as_structured_llm`\n",
"restaurant_obj = (\n",
" llm.as_structured_llm(Restaurant)\n",
" .complete(prompt_tmpl.format(city_name=\"Miami\"))\n",
" .raw\n",
")\n",
"# Option 2: Use `structured_predict`\n",
"# restaurant_obj = llm.structured_predict(Restaurant, prompt_tmpl, city_name=\"Miami\")"
]
},
{
Expand All @@ -691,6 +697,57 @@
"restaurant_obj"
]
},
{
"cell_type": "markdown",
"id": "b8e15b95-4fcb-4256-a96c-035dafa68c8c",
"metadata": {},
"source": [
"#### Structured Prediction with Streaming\n",
"\n",
"Any LLM wrapped with `as_structured_llm` supports streaming through `stream_chat`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ff9b820d-c091-4f1b-9fcb-997a13f62dd6",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'city': 'Miami', 'cuisine': 'Seafood', 'name': 'Miami Spice'}\n"
]
},
{
"data": {
"text/plain": [
"Restaurant(name='Miami Spice', city='Miami', cuisine='Seafood')"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from llama_index.core.llms import ChatMessage\n",
"from IPython.display import clear_output\n",
"from pprint import pprint\n",
"\n",
"input_msg = ChatMessage.from_str(\"Generate a restaurant in Miami\")\n",
"\n",
"sllm = llm.as_structured_llm(Restaurant)\n",
"stream_output = sllm.stream_chat([input_msg])\n",
"for partial_output in stream_output:\n",
" clear_output(wait=True)\n",
" pprint(partial_output.raw.dict())\n",
" restaurant_obj = partial_output.raw\n",
"\n",
"restaurant_obj"
]
},
{
"cell_type": "markdown",
"id": "5152a2b4-78e6-47a5-933d-f5186ec0f775",
Expand Down
82 changes: 76 additions & 6 deletions docs/docs/examples/llm/openai.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -420,7 +420,7 @@
"source": [
"## Structured Prediction\n",
"\n",
"An important use case for function calling is extracting structured objects. LlamaIndex provides an intuitive interface for this through `structured_predict` - simply define the target Pydantic class (can be nested), and given a prompt, we extract out the desired object."
"An important use case for function calling is extracting structured objects. LlamaIndex provides an intuitive interface for converting any LLM into a structured LLM - simply define the target Pydantic class (can be nested), and given a prompt, we extract out the desired object."
]
},
{
Expand All @@ -432,7 +432,15 @@
"source": [
"from llama_index.llms.openai import OpenAI\n",
"from llama_index.core.prompts import PromptTemplate\n",
"from pydantic import BaseModel\n",
"from pydantic.v1 import BaseModel\n",
"from typing import List\n",
"\n",
"\n",
"class MenuItem(BaseModel):\n",
" \"\"\"A menu item in a restaurant.\"\"\"\n",
"\n",
" course_name: str\n",
" is_vegetarian: bool\n",
"\n",
"\n",
"class Restaurant(BaseModel):\n",
Expand All @@ -441,15 +449,21 @@
" name: str\n",
" city: str\n",
" cuisine: str\n",
" menu_items: List[MenuItem]\n",
"\n",
"\n",
"llm = OpenAI(model=\"gpt-3.5-turbo\")\n",
"prompt_tmpl = PromptTemplate(\n",
" \"Generate a restaurant in a given city {city_name}\"\n",
")\n",
"restaurant_obj = llm.structured_predict(\n",
" Restaurant, prompt_tmpl, city_name=\"San Francisco\"\n",
")"
"# Option 1: Use `as_structured_llm`\n",
"restaurant_obj = (\n",
" llm.as_structured_llm(Restaurant)\n",
" .complete(prompt_tmpl.format(city_name=\"Dallas\"))\n",
" .raw\n",
")\n",
"# Option 2: Use `structured_predict`\n",
"# restaurant_obj = llm.structured_predict(Restaurant, prompt_tmpl, city_name=\"Miami\")"
]
},
{
Expand All @@ -461,7 +475,50 @@
{
"data": {
"text/plain": [
"Restaurant(name='Tasty Bites', city='San Francisco', cuisine='Italian')"
"Restaurant(name='Tasty Bites', city='Dallas', cuisine='Italian', menu_items=[MenuItem(course_name='Appetizer', is_vegetarian=True), MenuItem(course_name='Main Course', is_vegetarian=False), MenuItem(course_name='Dessert', is_vegetarian=True)])"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"restaurant_obj"
]
},
{
"cell_type": "markdown",
"id": "0c46b9ea-f6e0-4c4b-aaeb-9d77fcbb9f63",
"metadata": {},
"source": [
"#### Structured Prediction with Streaming\n",
"\n",
"Any LLM wrapped with `as_structured_llm` supports streaming through `stream_chat`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8d55877e-9313-4e94-b448-b8ef68ed0a6d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'city': 'Boston',\n",
" 'cuisine': 'American',\n",
" 'menu_items': [{'course_name': 'Appetizer', 'is_vegetarian': True},\n",
" {'course_name': 'Main Course', 'is_vegetarian': False},\n",
" {'course_name': 'Dessert', 'is_vegetarian': True}],\n",
" 'name': 'Boston Bites'}\n"
]
},
{
"data": {
"text/plain": [
"Restaurant(name='Boston Bites', city='Boston', cuisine='American', menu_items=[MenuItem(course_name='Appetizer', is_vegetarian=True), MenuItem(course_name='Main Course', is_vegetarian=False), MenuItem(course_name='Dessert', is_vegetarian=True)])"
]
},
"execution_count": null,
Expand All @@ -470,6 +527,19 @@
}
],
"source": [
"from llama_index.core.llms import ChatMessage\n",
"from IPython.display import clear_output\n",
"from pprint import pprint\n",
"\n",
"input_msg = ChatMessage.from_str(\"Generate a restaurant in Boston\")\n",
"\n",
"sllm = llm.as_structured_llm(Restaurant)\n",
"stream_output = sllm.stream_chat([input_msg])\n",
"for partial_output in stream_output:\n",
" clear_output(wait=True)\n",
" pprint(partial_output.raw.dict())\n",
" restaurant_obj = partial_output.raw\n",
"\n",
"restaurant_obj"
]
},
Expand Down
Loading

0 comments on commit 1e361b5

Please sign in to comment.