Skip to content

openai/openai-chatkit-advanced-samples

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

OpenAI ChatKit Examples

This repository collects scenario-driven ChatKit demos. Each example pairs a FastAPI backend with a Vite + React frontend, implementing a custom backend using ChatKit Python SDK and wiring it up with ChatKit.js client-side.

You can run the following examples:

  • Cat Lounge - caretaker for a virtual cat that helps improve energy, happiness, and cleanliness stats.
  • Customer Support – airline concierge with live itinerary data, timeline syncing, and domain-specific tools.
  • News Guide – Foxhollow Dispatch newsroom assistant with article search, @-mentions, and page-aware responses.
  • Metro Map – chat-driven metro planner with a React Flow network of lines and stations.

Quickstart

  1. Export OPENAI_API_KEY.
  2. Make sure uv is installed.
  3. Launch an example from the repo root, or with npm run start from the project directory:
Example Command for repo root Command for project directory URL
Cat Lounge npm run cat-lounge cd examples/cat-lounge && npm install && npm run start http://localhost:5170
Customer Support npm run customer-support cd examples/customer-support && npm install && npm start http://localhost:5171
News Guide npm run news-guide cd examples/news-guide && npm install && npm run start http://localhost:5172
Metro Map npm run metro-map cd examples/metro-map && npm install && npm run start http://localhost:5173

Feature index

Server tool calls to retrieve application data for inference

  • Cat Lounge:
    • Function tool get_cat_status (cat_agent.py) pulls the latest cat stats for the agent.
  • News Guide:
    • The agent leans on a suite of retrieval tools—list_available_tags_and_keywords, get_article_by_id, search_articles_by_tags/keywords/exact_text, and get_current_page—before responding, and uses show_article_list_widget to present results (news_agent.py).
    • Hidden context such as the featured landing page is normalized into agent input so summaries and recommendations stay grounded (news_agent.py).
  • Metro Map:
    • The metro agent syncs map data with get_map and surfaces line and station details via list_lines, list_stations, get_line_route, and get_station before giving directions (metro_map_agent.py).
    • show_line_selector presents the user a multiple-choice question using a widget.
    • Route-planning replies attach entity sources for the stations in the suggested path as annotations.

Client tool calls that mutate UI state

  • Cat Lounge:
    • Client tool update_cat_status is invoked by server tools feed_cat, play_with_cat, clean_cat, and speak_as_cat to sync UI state.
    • When invoked, it is handled client-side with the handleClientToolCall callback in ChatKitPanel.tsx.
  • News Guide:
    • The open_article widget action triggers client-side navigation to the selected story and forwards the action back to the server via sendCustomAction to follow up with context-aware prompts (ChatKitPanel.tsx).
  • Metro Map:
    • Client tools add_station (sent after the server adds a stop) and location_select_mode (sent after a line is chosen) update the metro map canvas (ChatKitPanel.tsx).
    • Inference is deliberately skipped after the location_select_mode client tool call output is sent to the server; the respond method on the server early returns when the last item in the thread is the location_select_mode client tool call (server.py).
    • The location_select_mode client tool call is streamed within the server action handler (server.py).

Page-aware model responses

  • News Guide:
    • The ChatKit client forwards the currently open article id in an article-id header so the backend can scope responses to “this page” (ChatKitPanel.tsx).
    • The server reads that request context and exposes get_current_page so the agent can load full content without asking the user to paste it (main.py, news_agent.py).

Progress updates

  • News Guide:
    • Retrieval tools stream ProgressUpdateEvent messages while searching tags, authors, keywords, exact text, or loading the current page so the UI surfaces “Searching…”/“Loading…” states (news_agent.py).
    • The event finder emits progress as it scans dates, days of week, or keywords to keep users informed during longer lookups (event_finder_agent.py).
  • Metro Map:
    • The metro agent emits a quick sync update when it loads the line data via get_map (metro_map_agent.py).

Widgets without actions

Widgets with actions

  • Cat Lounge:
    • Server tool suggest_cat_names streams a widget with action configs that specify cats.select_name and cats.more_names client-handled actions.
    • When the user clicks the widget, these actions are handled with the handleWidgetAction callback in ChatKitPanel.tsx.
  • News Guide:
  • Metro Map:

Server-handled widget actions

  • Cat Lounge:
    • The cats.select_name action is also handled server-side to reflect updates to data and stream back an updated version of the name suggestions widget in server.py.
    • It is invoked using chatkit.sendAction() from handleWidgetAction callback in ChatKitPanel.tsx.
  • News Guide:
    • The view_event_details action is processed server-side to update the timeline widget with expanded descriptions without a round trip to the model (server.py).
  • Metro Map:
    • The line.select action is handled server-side to stream an updated widget, add a <LINE_SELECTED> hidden context item to thread, stream an assistant message to ask the user whether to add the station at the line’s start or end, and trigger the location_select_mode client tool call for the UI to sync (server.py).

Annotations

  • Metro Map:
    • The plan_route tool renders each station in a planned route as an entity source on the assistant message; the client’s entity click handler pans the React Flow canvas to the clicked station (ChatKitPanel.tsx, metro_map_agent.py).

Thread titles

  • Cat Lounge:
    • After the user names the cat, the set_cat_name tool locks in the name and updates the thread title to {name}’s Lounge before saving it (cat_agent.py).
  • News Guide:
    • The title_agent runs on the first user message to generate a short newsroom-friendly title when none exists (server.py, title_agent.py).
  • Metro Map:
    • The metro server uses a dedicated title_agent to set a brief metro-planning title on the first turn and persists it to thread metadata (server.py, title_agent.py).

Entity tags (@-mentions)

  • News Guide:
    • Entity search and previews power @-mentions for articles/authors in the composer and render hover previews via /articles/tags (ChatKitPanel.tsx, main.py).
    • Tagged entities are converted into model-readable markers so the agent can fetch the right records (<ARTICLE_REFERENCE> / <AUTHOR_REFERENCE>) (thread_item_converter.py).
    • Article reference tags are resolved into full articles via the instructed get_article_by_id tool before the agent cites details (news_agent.py).
  • Metro Map:
    • The composer’s entity search lists stations so users can @-mention them; clicking a tag also focuses the station on the canvas (ChatKitPanel.tsx).
    • Tagged stations are converted into <STATION_TAG> blocks with full line metadata so the agent can answer without another lookup (thread_item_converter.py, server.py).

Tool choice (composer menu)

  • News Guide:
    • The ChatKit client is configured with a composer.tools option that specifies options in the composer menu (ChatKitPanel.tsx)
    • Composer tool buttons let users force specific agents (event_finder, puzzle), setting tool_choice on the request (config.ts).
    • The backend routes these tool choices to specialized agents before falling back to the News Guide agent (server.py).

Custom header actions

  • Metro Map:
    • The chat header uses a right-side icon toggle (dark-mode / light-mode) to flip the app’s color scheme client-side (ChatKitPanel.tsx).