Chatbot Examples (#8966)

* examples

* examples

* first pass

* remove comments

* remove comments

* add changeset

* Fix chatinterface e2e test (#9104)

* Refactor test

* comment

* Fix image

* add changeset

* add changeset

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* fix version + pkg name (#9110)

* fix version + pkg name

* add changeset

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* fix dev (#9115)

* fix asset locations

* fix changeset

* Be able to set optimizeDeps options in gradio.config.js (#9091)

* Add code

* add changeset

* build

* Remove unused import

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* Better text styling on docs (#9108)

* margin and size

* add changeset

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* setup npm-previews of all packages (#9118)

* add workflow

* fix pkg jsons

* workflow name

* add changeset

* fix

* add changeset

* fix build command

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* Fix chatinterface multimodal bug (#9119)

* Add test

* add changeset

* comments

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* chatbot examples

* examples changes

* chatinterface

* chatinterface

* pr fixes

* remove html demo change

* suggestion width

* type fixes

* format

* comment our examples test

* remove cache

* comment example caching test

* bug fix

* bug fix

* format

* type fix

* Proposal: remove `gr.make_waveform` and remove `matplotlib` as a dependency (#9312)

* remove matplotlib

* add changeset

* remove tests, demo

* Fix the Lite worker to set the matplotlib backend engine only when the matplotlib package is installed

* add changeset

* Fix comment

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>
Co-authored-by: Yuichiro Tachibana (Tsuchiya) <t.yic.yt@gmail.com>

* Dont move files to cache automatically in chatbot postprocess (#9303)

* Fix

* add changeset

* Add code

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>

* test fix

* format

* changes

* update guide

* cache examples

* add changeset

* format

* changes

* changes

* changes

* changes

* changes

* changes

* format

* fixes

* test chat interface fixes

---------

Co-authored-by: gradio-pr-bot <gradio-pr-bot@users.noreply.github.com>
Co-authored-by: Freddy Boulton <alfonsoboulton@gmail.com>
Co-authored-by: pngwn <hello@pngwn.io>
Co-authored-by: Ali Abdalla <ali.si3luwa@gmail.com>
Co-authored-by: Abubakar Abid <abubakar@huggingface.co>
Co-authored-by: Yuichiro Tachibana (Tsuchiya) <t.yic.yt@gmail.com>
Co-authored-by: Ali Abid <aliabid94@gmail.com>

.changeset/dark-moose-shine.md

@@ -0,0 +1,8 @@
+---
+"@gradio/chatbot": minor
+"@gradio/multimodaltextbox": minor
+"@gradio/textbox": minor
+"gradio": minor
+---
+
+feat:Chatbot Examples

demo/chatbot_multimodal/run.ipynb

@@ -1 +1 @@
-{"cells": [{"cell_type": "markdown", "id": "302934307671667531413257853548643485645", "metadata": {}, "source": ["# Gradio Demo: chatbot_multimodal"]}, {"cell_type": "code", "execution_count": null, "id": "272996653310673477252411125948039410165", "metadata": {}, "outputs": [], "source": ["!pip install -q gradio plotly"]}, {"cell_type": "code", "execution_count": null, "id": "288918539441861185822528903084949547379", "metadata": {}, "outputs": [], "source": ["# Downloading files from the demo repo\n", "import os\n", "os.mkdir('files')\n", "!wget -q -O files/avatar.png https://github.com/gradio-app/gradio/raw/main/demo/chatbot_multimodal/files/avatar.png\n", "!wget -q https://github.com/gradio-app/gradio/raw/main/demo/chatbot_multimodal/tuples_testcase.py"]}, {"cell_type": "code", "execution_count": null, "id": "44380577570523278879349135829904343037", "metadata": {}, "outputs": [], "source": ["import gradio as gr\n", "import time\n", "\n", "# Chatbot demo with multimodal input (text, markdown, LaTeX, code blocks, image, audio, & video). Plus shows support for streaming text.\n", "\n", "def print_like_dislike(x: gr.LikeData):\n", "    print(x.index, x.value, x.liked)\n", "\n", "def add_message(history, message):\n", "    for x in message[\"files\"]:\n", "        history.append({\"role\": \"user\", \"content\": {\"path\": x}})\n", "    if message[\"text\"] is not None:\n", "        history.append({\"role\": \"user\", \"content\": message[\"text\"]})\n", "    return history, gr.MultimodalTextbox(value=None, interactive=False)\n", "\n", "def bot(history: list):\n", "    response = \"**That's cool!**\"\n", "    history.append({\"role\": \"assistant\", \"content\": \"\"})\n", "    for character in response:\n", "        history[-1]['content'] += character\n", "        time.sleep(0.05)\n", "        yield history\n", "\n", "with gr.Blocks() as demo:\n", "    chatbot = gr.Chatbot(\n", "        elem_id=\"chatbot\",\n", "        bubble_full_width=False,\n", "        type=\"messages\"\n", "    )\n", "\n", "    chat_input = gr.MultimodalTextbox(interactive=True,\n", "                                      file_count=\"multiple\",\n", "                                      placeholder=\"Enter message or upload file...\", show_label=False)\n", "\n", "    chat_msg = chat_input.submit(add_message, [chatbot, chat_input], [chatbot, chat_input])\n", "    bot_msg = chat_msg.then(bot, chatbot, chatbot, api_name=\"bot_response\")\n", "    bot_msg.then(lambda: gr.MultimodalTextbox(interactive=True), None, [chat_input])\n", "\n", "    chatbot.like(print_like_dislike, None, None, like_user_message=True)\n", "\n", "if __name__ == \"__main__\":\n", "    demo.launch()\n"]}], "metadata": {}, "nbformat": 4, "nbformat_minor": 5}
+{"cells": [{"cell_type": "markdown", "id": "302934307671667531413257853548643485645", "metadata": {}, "source": ["# Gradio Demo: chatbot_multimodal"]}, {"cell_type": "code", "execution_count": null, "id": "272996653310673477252411125948039410165", "metadata": {}, "outputs": [], "source": ["!pip install -q gradio "]}, {"cell_type": "code", "execution_count": null, "id": "288918539441861185822528903084949547379", "metadata": {}, "outputs": [], "source": ["# Downloading files from the demo repo\n", "import os\n", "!wget -q https://github.com/gradio-app/gradio/raw/main/demo/chatbot_multimodal/tuples_testcase.py"]}, {"cell_type": "code", "execution_count": null, "id": "44380577570523278879349135829904343037", "metadata": {}, "outputs": [], "source": ["import gradio as gr\n", "import time\n", "\n", "# Chatbot demo with multimodal input (text, markdown, LaTeX, code blocks, image, audio, & video). Plus shows support for streaming text.\n", "\n", "def print_like_dislike(x: gr.LikeData):\n", "    print(x.index, x.value, x.liked)\n", "\n", "def add_message(history, message):\n", "    for x in message[\"files\"]:\n", "        history.append({\"role\": \"user\", \"content\": {\"path\": x}})\n", "    if message[\"text\"] is not None:\n", "        history.append({\"role\": \"user\", \"content\": message[\"text\"]})\n", "    return history, gr.MultimodalTextbox(value=None, interactive=False)\n", "\n", "def bot(history: list):\n", "    response = \"**That's cool!**\"\n", "    history.append({\"role\": \"assistant\", \"content\": \"\"})\n", "    for character in response:\n", "        history[-1]['content'] += character\n", "        time.sleep(0.05)\n", "        yield history\n", "\n", "with gr.Blocks() as demo:\n", "    chatbot = gr.Chatbot(\n", "        elem_id=\"chatbot\",\n", "        bubble_full_width=False,\n", "        type=\"messages\"\n", "    )\n", "\n", "    chat_input = gr.MultimodalTextbox(interactive=True,\n", "                                      file_count=\"multiple\",\n", "                                      placeholder=\"Enter message or upload file...\", show_label=False)\n", "\n", "    chat_msg = chat_input.submit(add_message, [chatbot, chat_input], [chatbot, chat_input])\n", "    bot_msg = chat_msg.then(bot, chatbot, chatbot, api_name=\"bot_response\")\n", "    bot_msg.then(lambda: gr.MultimodalTextbox(interactive=True), None, [chat_input])\n", "\n", "    chatbot.like(print_like_dislike, None, None, like_user_message=True)\n", "\n", "if __name__ == \"__main__\":\n", "    demo.launch()\n"]}], "metadata": {}, "nbformat": 4, "nbformat_minor": 5}

demo/chatbot_suggestions/run.ipynb

@@ -0,0 +1 @@
+{"cells": [{"cell_type": "markdown", "id": "302934307671667531413257853548643485645", "metadata": {}, "source": ["# Gradio Demo: chatbot_suggestions"]}, {"cell_type": "code", "execution_count": null, "id": "272996653310673477252411125948039410165", "metadata": {}, "outputs": [], "source": ["!pip install -q gradio "]}, {"cell_type": "code", "execution_count": null, "id": "288918539441861185822528903084949547379", "metadata": {}, "outputs": [], "source": ["# Downloading files from the demo repo\n", "import os\n", "os.mkdir('files')\n", "!wget -q -O files/avatar.png https://github.com/gradio-app/gradio/raw/main/demo/chatbot_suggestions/files/avatar.png\n", "!wget -q -O files/cantina.wav https://github.com/gradio-app/gradio/raw/main/demo/chatbot_suggestions/files/cantina.wav"]}, {"cell_type": "code", "execution_count": null, "id": "44380577570523278879349135829904343037", "metadata": {}, "outputs": [], "source": ["import gradio as gr\n", "import os\n", "# Multimodal Chatbot demo that shows support for suggestions (example messages shown within the chatbot).\n", "\n", "def print_like_dislike(x: gr.LikeData):\n", "    print(x.index, x.value, x.liked)\n", "\n", "def add_message(history, message):\n", "    for x in message[\"files\"]:\n", "        history.append(((x,), None))\n", "    if message[\"text\"] is not None:\n", "        history.append((message[\"text\"], None))\n", "    return history, gr.MultimodalTextbox(value=None, interactive=False)\n", "\n", "def append_suggestion_message(x: gr.SelectData, history):\n", "    if x.value[\"text\"] is not None:\n", "        history.append((x.value[\"text\"], None))\n", "    if \"files\" in x.value:\n", "        if isinstance(x.value[\"files\"], list):\n", "            for file in x.value[\"files\"]:\n", "                history.append((file, None))\n", "        else:\n", "            history.append((x.value[\"files\"], None))\n", "    return history\n", "\n", "def respond(history):\n", "    history[-1][1] = \"Cool!\"\n", "    return history\n", "\n", "with gr.Blocks(fill_height=True) as demo:\n", "    chatbot = gr.Chatbot(\n", "        elem_id=\"chatbot\",\n", "        bubble_full_width=False,\n", "        scale=1,\n", "        placeholder='<h1 style=\"font-weight: bold; color: #FFFFFF; text-align: center; font-size: 48px; font-family: Arial, sans-serif;\">Welcome to Gradio!</h1>',\n", "        suggestions=[{\"icon\": os.path.join(os.path.abspath(''), \"files/avatar.png\"), \"display_text\": \"Display Text Here!\", \"text\": \"Try this example with this audio.\", \"files\": [os.path.join(os.path.abspath(''), \"files/cantina.wav\")]},\n", "                     {\"text\": \"Try this example with this image.\", \"files\": [os.path.join(os.path.abspath(''), \"files/avatar.png\")]},\n", "                     {\"text\": \"This is just text, no files!\"},\n", "                     {\"text\": \"Try this example with this image.\", \"files\": [os.path.join(os.path.abspath(''), \"files/avatar.png\"), os.path.join(os.path.abspath(''), \"files/avatar.png\")]},\n", "                     {\"text\": \"Try this example with this Audio.\", \"files\": [os.path.join(os.path.abspath(''), \"files/cantina.wav\")]}]\n", "    )\n", "\n", "    chat_input = gr.MultimodalTextbox(interactive=True,\n", "                                      file_count=\"multiple\",\n", "                                      placeholder=\"Enter message or upload file...\", show_label=False)\n", "\n", "    chat_msg = chat_input.submit(add_message, [chatbot, chat_input], [chatbot, chat_input])\n", "    bot_msg = chat_msg.then(respond, chatbot, chatbot, api_name=\"bot_response\")\n", "    bot_msg.then(lambda: gr.MultimodalTextbox(interactive=True), None, [chat_input])\n", "\n", "    chatbot.like(print_like_dislike, None, None)\n", "    chatbot.suggestion_select(append_suggestion_message, [chatbot], [chatbot]).then(respond, chatbot, chatbot, api_name=\"respond\")\n", "\n", "if __name__ == \"__main__\":\n", "    demo.launch()\n"]}], "metadata": {}, "nbformat": 4, "nbformat_minor": 5}

demo/chatbot_suggestions/run.py

@@ -0,0 +1,55 @@
+import gradio as gr
+import os
+# Multimodal Chatbot demo that shows support for suggestions (example messages shown within the chatbot).
+
+def print_like_dislike(x: gr.LikeData):
+    print(x.index, x.value, x.liked)
+
+def add_message(history, message):
+    for x in message["files"]:
+        history.append(((x,), None))
+    if message["text"] is not None:
+        history.append((message["text"], None))
+    return history, gr.MultimodalTextbox(value=None, interactive=False)
+
+def append_suggestion_message(x: gr.SelectData, history):
+    if x.value["text"] is not None:
+        history.append((x.value["text"], None))
+    if "files" in x.value:
+        if isinstance(x.value["files"], list):
+            for file in x.value["files"]:
+                history.append((file, None))
+        else:
+            history.append((x.value["files"], None))
+    return history
+
+def respond(history):
+    history[-1][1] = "Cool!"
+    return history
+
+with gr.Blocks(fill_height=True) as demo:
+    chatbot = gr.Chatbot(
+        elem_id="chatbot",
+        bubble_full_width=False,
+        scale=1,
+        placeholder='<h1 style="font-weight: bold; color: #FFFFFF; text-align: center; font-size: 48px; font-family: Arial, sans-serif;">Welcome to Gradio!</h1>',
+        suggestions=[{"icon": os.path.join(os.path.dirname(__file__), "files/avatar.png"), "display_text": "Display Text Here!", "text": "Try this example with this audio.", "files": [os.path.join(os.path.dirname(__file__), "files/cantina.wav")]},
+                     {"text": "Try this example with this image.", "files": [os.path.join(os.path.dirname(__file__), "files/avatar.png")]},
+                     {"text": "This is just text, no files!"},
+                     {"text": "Try this example with this image.", "files": [os.path.join(os.path.dirname(__file__), "files/avatar.png"), os.path.join(os.path.dirname(__file__), "files/avatar.png")]},
+                     {"text": "Try this example with this Audio.", "files": [os.path.join(os.path.dirname(__file__), "files/cantina.wav")]}]
+    )
+
+    chat_input = gr.MultimodalTextbox(interactive=True,
+                                      file_count="multiple",
+                                      placeholder="Enter message or upload file...", show_label=False)
+
+    chat_msg = chat_input.submit(add_message, [chatbot, chat_input], [chatbot, chat_input])
+    bot_msg = chat_msg.then(respond, chatbot, chatbot, api_name="bot_response")
+    bot_msg.then(lambda: gr.MultimodalTextbox(interactive=True), None, [chat_input])
+
+    chatbot.like(print_like_dislike, None, None)
+    chatbot.suggestion_select(append_suggestion_message, [chatbot], [chatbot]).then(respond, chatbot, chatbot, api_name="respond")
+
+if __name__ == "__main__":
+    demo.launch()

demo/chatinterface_multimodal/run.ipynb

@@ -1 +1 @@
-{"cells": [{"cell_type": "markdown", "id": "302934307671667531413257853548643485645", "metadata": {}, "source": ["# Gradio Demo: chatinterface_multimodal"]}, {"cell_type": "code", "execution_count": null, "id": "272996653310673477252411125948039410165", "metadata": {}, "outputs": [], "source": ["!pip install -q gradio "]}, {"cell_type": "code", "execution_count": null, "id": "288918539441861185822528903084949547379", "metadata": {}, "outputs": [], "source": ["import gradio as gr\n", "\n", "def echo(message, history):\n", "    return message[\"text\"]\n", "\n", "demo = gr.ChatInterface(\n", "    fn=echo,\n", "    type=\"messages\",\n", "    examples=[{\"text\": \"hello\"}, {\"text\": \"hola\"}, {\"text\": \"merhaba\"}],\n", "    title=\"Echo Bot\",\n", "    multimodal=True,\n", ")\n", "demo.launch()\n"]}], "metadata": {}, "nbformat": 4, "nbformat_minor": 5}
+{"cells": [{"cell_type": "markdown", "id": "302934307671667531413257853548643485645", "metadata": {}, "source": ["# Gradio Demo: chatinterface_multimodal"]}, {"cell_type": "code", "execution_count": null, "id": "272996653310673477252411125948039410165", "metadata": {}, "outputs": [], "source": ["!pip install -q gradio "]}, {"cell_type": "code", "execution_count": null, "id": "288918539441861185822528903084949547379", "metadata": {}, "outputs": [], "source": ["# Downloading files from the demo repo\n", "import os\n", "os.mkdir('files')\n", "!wget -q -O files/avatar.png https://github.com/gradio-app/gradio/raw/main/demo/chatinterface_multimodal/files/avatar.png\n", "!wget -q -O files/cantina.wav https://github.com/gradio-app/gradio/raw/main/demo/chatinterface_multimodal/files/cantina.wav"]}, {"cell_type": "code", "execution_count": null, "id": "44380577570523278879349135829904343037", "metadata": {}, "outputs": [], "source": ["import gradio as gr\n", "\n", "def echo(message, history):\n", "    return message[\"text\"]\n", "\n", "demo = gr.ChatInterface(\n", "    fn=echo,\n", "    type=\"messages\",\n", "    examples=[{\"text\": \"hello\"}, {\"text\": \"hola\"}, {\"text\": \"merhaba\"}],\n", "    title=\"Echo Bot\",\n", "    multimodal=True,\n", ")\n", "demo.launch()\n"]}], "metadata": {}, "nbformat": 4, "nbformat_minor": 5}