Add guides for msg format and llm agents #8750

freddyaboulton · 2024-07-11T11:21:40Z

Description

Also renames msg_format to type

🎯 PRs Should Target Issues

Before your create a PR, please check to see if there is an existing issue for this change. If not, please create an issue before you create this PR, unless the fix is very small.

Not adhering to this guideline will result in the PR being closed.

Tests

PRs will only be merged if tests pass on CI. To run the tests locally, please set up your Gradio environment locally and run the tests: bash scripts/run_all_tests.sh
You may need to run the linters: bash scripts/format_backend.sh and bash scripts/format_frontend.sh

gradio-pr-bot · 2024-07-11T11:22:29Z

🪼 branch checks and previews

•	Name	Status	URL
	Spaces	ready!	Spaces preview
	Website	ready!	Website preview
	Storybook	ready!	Storybook preview
🦄	Changes	detected!	Details

Install Gradio from this PR

pip install https://gradio-builds.s3.amazonaws.com/7eacc4ceec7ea611b44267edbbbe9d054ad557b2/gradio-4.37.2-py3-none-any.whl

Install Gradio Python Client from this PR

pip install "gradio-client @ git+https://github.com/gradio-app/gradio@7eacc4ceec7ea611b44267edbbbe9d054ad557b2#subdirectory=client/python"

Install Gradio JS Client from this PR

npm install https://gradio-builds.s3.amazonaws.com/7eacc4ceec7ea611b44267edbbbe9d054ad557b2/gradio-client-1.2.1.tgz

gradio-pr-bot · 2024-07-11T11:22:39Z

🦄 change detected

This Pull Request includes changes to the following packages.

Package	Version
`@gradio/chatbot`	`minor`
`@gradio/tootils`	`minor`
`gradio`	`minor`
`website`	`minor`

With the following changelog entry.

Support message format in chatbot 💬

gr.Chatbot and gr.ChatInterface now support the Messages API, which is fully compatible with LLM API providers such as Hugging Face Text Generation Inference, OpenAI's chat completions API, and Llama.cpp server.

Building Gradio applications around these LLM solutions is now even easier!

gr.Chatbot and gr.ChatInterface now have a type parameter that can accept two values - 'tuples' and 'messages'. If set to 'tuples', the default chatbot data format is expected. If set to 'messages', a list of dictionaries with content and role keys is expected. See below -

def chat_greeter(msg, history):
    history.append({"role": "assistant", "content": "Hello!"})
    return history

Additionally, gradio now exposes a gr.ChatMessage dataclass you can use for IDE type hints and auto completion.

Tool use in Chatbot 🛠️

The Gradio Chatbot can now natively display tool usage and intermediate thoughts common in Agent and chain-of-thought workflows!

If you are using the new "messages" format, simply add a metadata key with a dictionary containing a title key and value. This will display the assistant message in an expandable message box to show the result of a tool or intermediate step.

import gradio as gr
from gradio import ChatMessage
import time

def generate_response(history):
    history.append(ChatMessage(role="user", content="What is the weather in San Francisco right now?"))
    yield history
    time.sleep(0.25)
    history.append(ChatMessage(role="assistant",
                               content="In order to find the current weather in San Francisco, I will need to use my weather tool.")
                               )
    yield history
    time.sleep(0.25)

    history.append(ChatMessage(role="assistant",
                               content="API Error when connecting to weather service.",
                              metadata={"title": "💥 Error using tool 'Weather'"})
                  )
    yield history
    time.sleep(0.25)

    history.append(ChatMessage(role="assistant",
                               content="I will try again",
                              ))
    yield history
    time.sleep(0.25)

    history.append(ChatMessage(role="assistant",
                               content="Weather 72 degrees Fahrenheit with 20% chance of rain.",
                                metadata={"title": "🛠️ Used tool 'Weather'"}
                              ))
    yield history
    time.sleep(0.25)

    history.append(ChatMessage(role="assistant",
                               content="Now that the API succeeded I can complete my task.",
                              ))
    yield history
    time.sleep(0.25)

    history.append(ChatMessage(role="assistant",
                               content="It's a sunny day in San Francisco with a current temperature of 72 degrees Fahrenheit and a 20% chance of rain. Enjoy the weather!",
                              ))
    yield history


with gr.Blocks() as demo:
    chatbot  = gr.Chatbot(type="messages")
    button = gr.Button("Get San Francisco Weather")
    button.click(generate_response, chatbot, chatbot)

if __name__ == "__main__":
    demo.launch()

⚠️ The changeset file for this pull request has been modified manually, so the changeset generation bot has been disabled. To go back into automatic mode, delete the changeset file.

Something isn't right?

Maintainers can change the version label to modify the version bump.
If the bot has failed to detect any changes, or if this pull request needs to update multiple packages to different versions or requires a more comprehensive changelog entry, maintainers can update the changelog file directly.

freddyaboulton · 2024-07-11T12:02:37Z

Need to fix the deployed spaces but content is good for a review. @aliabd why is the build failing on the redirect? I followed the pattern in creating-a-custom-chatbot-with-blocks.md

dawoodkhan82 · 2024-07-11T18:39:04Z

demo/agent_chatbot/run.py

+                         avatar_images=(None, "https://em-content.zobj.net/source/twitter/53/robot-face_1f916.png"))
+    text_input = gr.Textbox(lines=1, label="Chat Message")
+    text_input.submit(interact_with_agent, [text_input, chatbot], [chatbot])
+


I think text_input needs to be cleared here after submit

Done! Good call

dawoodkhan82 · 2024-07-11T18:39:50Z

guides/05_chatbots/03-messages-format.md

+def chat_function(message, history):
+    history.append(ChatMessage(role="user", content=message))
+    history.append(ChatMessage(role="assistant", content="Hello, how can I help you?"))
+    return history


dawoodkhan82

Left a small comment, but guides look good! Thanks @freddyaboulton

aliabd

Everything looks good apart from my comments on website build.

Thanks @freddyaboulton!

aliabd · 2024-07-12T01:25:21Z

guides/05_chatbots/03-messages-format.md

@@ -0,0 +1,97 @@
+# Using the Messages data format


The name of this file should be 03_messages-format.md (the numerical prefix is followed by an underscore not a dash)

🤦 Thank you!

aliabd · 2024-07-12T01:32:31Z

guides/05_chatbots/04_agents-and-tool-usage.md

+Tags: LLM, AGENTS, CHAT
+Related spaces: https://huggingface.co/spaces/gradio/agent_chatbot, https://huggingface.co/spaces/gradio/langchain-agent
+
+The Gradio Chatbot can natively display intermediate thoughts and tool usage. This makes it perfect for creating UIs for LLM agents. This guide will show you how. Before we begin, familiarize yourself with the `messages` chatbot data format documented in this [guide](/guides/05_chatbots/messages-format).


This is what's currently breaking the build. Guide urls are just /guides/ there is no folder (chatbots in this case), and no numerical prefix. But also you're linking to a guide that doesn't exist on the release yet (because there hasn't been a release yet) so you need to link to the main url until then. The right way is to use a relative url so it finds the guide if it's on main or pip.

Suggested change

The Gradio Chatbot can natively display intermediate thoughts and tool usage. This makes it perfect for creating UIs for LLM agents. This guide will show you how. Before we begin, familiarize yourself with the `messages` chatbot data format documented in this [guide](/guides/05_chatbots/messages-format).

The Gradio Chatbot can natively display intermediate thoughts and tool usage. This makes it perfect for creating UIs for LLM agents. This guide will show you how. Before we begin, familiarize yourself with the `messages` chatbot data format documented in this [guide](./messages-format).

Done - thanks @aliabd !

This reverts commit 7181f0e, reversing changes made to c2bdb95.

Add guides

9295019

freddyaboulton added the v: patch A change that requires a patch release label Jul 11, 2024

Merge branch 'main' into agent-guide

60463e5

add changeset

2ec2d32

freddyaboulton added 3 commits July 11, 2024 13:29

Add code

8bd61f4

Add code

c09fda8

Add notebook

f76726f

rename msg_format to type

d084d9f

freddyaboulton force-pushed the agent-guide branch from 13deb6e to d084d9f Compare July 11, 2024 15:00

freddyaboulton added 4 commits July 11, 2024 17:01

Fix docs

6601305

notebooks

803a207

Merge branch 'main' into agent-guide

d7f7799

missing link

f71cee2

freddyaboulton marked this pull request as ready for review July 11, 2024 15:17

freddyaboulton requested review from aliabid94, dawoodkhan82, abidlabs, pngwn, hannahblair and aliabd July 11, 2024 15:17

Merge branch 'main' into agent-guide

0237515

dawoodkhan82 reviewed Jul 11, 2024

View reviewed changes

dawoodkhan82 self-requested a review July 11, 2024 18:40

dawoodkhan82 approved these changes Jul 11, 2024

View reviewed changes

aliabd approved these changes Jul 12, 2024

View reviewed changes

Update guides

c2bdb95

Merge branch 'main' into agent-guide

7181f0e

freddyaboulton enabled auto-merge (squash) July 12, 2024 10:50

Revert "Merge branch 'main' into agent-guide"

7eacc4c

This reverts commit 7181f0e, reversing changes made to c2bdb95.

freddyaboulton merged commit 5e36144 into main Jul 12, 2024
9 checks passed

freddyaboulton deleted the agent-guide branch July 12, 2024 11:53

pngwn mentioned this pull request Jul 12, 2024

chore: update versions #8663

Merged

whitphx mentioned this pull request Jul 17, 2024

Refactoring <gradio-lite /> component making the code simpler and fixing a Playground mode bug #8818

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add guides for msg format and llm agents #8750

Add guides for msg format and llm agents #8750

freddyaboulton commented Jul 11, 2024 •

edited

Loading

gradio-pr-bot commented Jul 11, 2024 •

edited

Loading

gradio-pr-bot commented Jul 11, 2024 •

edited

Loading

Support message format in chatbot 💬

Tool use in Chatbot 🛠️

Something isn't right?

freddyaboulton commented Jul 11, 2024

dawoodkhan82 Jul 11, 2024

freddyaboulton Jul 12, 2024

dawoodkhan82 Jul 11, 2024

dawoodkhan82 left a comment

aliabd left a comment

aliabd Jul 12, 2024

freddyaboulton Jul 12, 2024

aliabd Jul 12, 2024

freddyaboulton Jul 12, 2024

	The Gradio Chatbot can natively display intermediate thoughts and tool usage. This makes it perfect for creating UIs for LLM agents. This guide will show you how. Before we begin, familiarize yourself with the `messages` chatbot data format documented in this [guide](/guides/05_chatbots/messages-format).
	The Gradio Chatbot can natively display intermediate thoughts and tool usage. This makes it perfect for creating UIs for LLM agents. This guide will show you how. Before we begin, familiarize yourself with the `messages` chatbot data format documented in this [guide](./messages-format).

Add guides for msg format and llm agents #8750

Add guides for msg format and llm agents #8750

Conversation

freddyaboulton commented Jul 11, 2024 • edited Loading

Description

🎯 PRs Should Target Issues

Tests

gradio-pr-bot commented Jul 11, 2024 • edited Loading

🪼 branch checks and previews

gradio-pr-bot commented Jul 11, 2024 • edited Loading

🦄 change detected

This Pull Request includes changes to the following packages.

With the following changelog entry.

Support message format in chatbot 💬

Tool use in Chatbot 🛠️

Something isn't right?

freddyaboulton commented Jul 11, 2024

dawoodkhan82 Jul 11, 2024

Choose a reason for hiding this comment

freddyaboulton Jul 12, 2024

Choose a reason for hiding this comment

dawoodkhan82 Jul 11, 2024

Choose a reason for hiding this comment

dawoodkhan82 left a comment

Choose a reason for hiding this comment

aliabd left a comment

Choose a reason for hiding this comment

aliabd Jul 12, 2024

Choose a reason for hiding this comment

freddyaboulton Jul 12, 2024

Choose a reason for hiding this comment

aliabd Jul 12, 2024

Choose a reason for hiding this comment

freddyaboulton Jul 12, 2024

Choose a reason for hiding this comment

freddyaboulton commented Jul 11, 2024 •

edited

Loading

gradio-pr-bot commented Jul 11, 2024 •

edited

Loading

gradio-pr-bot commented Jul 11, 2024 •

edited

Loading