Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 135 additions & 0 deletions examples/widget_test/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
# OpenAI Widget Demo

This directory contains a demonstration of FastMCP's OpenAI widget support using the Pizzaz mapping library.

## What's Here

- **`pizzaz_server.py`**: Demo server with three widget examples showing different return patterns
- **`test_widgets.py`**: Test script that validates the auto-transformation works correctly

## Setup

This uses a local virtual environment with an editable install of FastMCP:

```bash
# Already done, but for reference:
cd examples/widget_test
uv venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
uv pip install -e ../..
```

## Running the Demo

### Test the Widgets Locally

```bash
source .venv/bin/activate
python test_widgets.py
```

This will test all three widget patterns:
- **dict return**: Structured data only (no narrative text)
- **str return**: Narrative text only (no structured data)
- **tuple[str, dict] return**: Both narrative text and structured data

### Run the Server

```bash
source .venv/bin/activate
python pizzaz_server.py
```

The server will start on `http://0.0.0.0:8080`.

### Inspect the Server

```bash
source .venv/bin/activate
fastmcp inspect pizzaz_server.py
```

Use `--format fastmcp` to see the full JSON including OpenAI metadata.

## Testing with ChatGPT

To test the widgets with ChatGPT:

1. Expose your server with ngrok:
```bash
ngrok http 8080
```

2. In ChatGPT, go to Settings → Developer Mode and add your server

3. Try prompts like:
- "Show me a pizza map for pepperoni"
- "Track pizza order 12345"
- "What's the pizza status?"

## Widget Examples

### 1. Pizza Map (dict return)

Returns structured data only, which the widget renders:

```python
@app.ui.openai.widget(
name="pizza-map",
template_uri="ui://widget/pizza-map.html",
html=PIZZAZ_HTML,
invoking="Hand-tossing a map",
invoked="Served a fresh map",
)
def show_pizza_map(topping: str) -> dict:
return {"pizza_topping": topping.strip()}
```

### 2. Pizza Tracker (tuple return)

Returns both narrative text and structured data:

```python
@app.ui.openai.widget(
name="pizza-tracker",
template_uri="ui://widget/pizza-tracker.html",
html=PIZZAZ_HTML,
invoking="Tracking your pizza",
invoked="Pizza located!",
)
def track_pizza(order_id: str) -> tuple[str, dict]:
narrative = f"Tracking pizza order {order_id}..."
data = {"order_id": order_id, "status": "out_for_delivery"}
return narrative, data
```

### 3. Pizza Status (str return)

Returns text only, no structured data:

```python
@app.ui.openai.widget(
name="pizza-status",
template_uri="ui://widget/pizza-status.html",
html=PIZZAZ_HTML,
)
def pizza_status() -> str:
return "Pizza ovens are hot and ready! 🍕"
```

## How It Works

The `@app.ui.openai.widget` decorator automatically:

1. **Registers the HTML** as an MCP resource with MIME type `text/html+skybridge`
2. **Adds OpenAI metadata** to the tool including:
- `openai/outputTemplate`: Points to the widget HTML
- `openai/toolInvocation/invoking` and `invoked`: Status messages
- `openai.com/widget`: Embedded widget resource
- CSP configuration for security
3. **Auto-transforms return values** to OpenAI format:
- `dict` → `{"content": [], "structuredContent": {...}}`
- `str` → `{"content": [{"type": "text", "text": "..."}], "structuredContent": {}}`
- `tuple[str, dict]` → Both content and structuredContent

You don't need to manually call `build_widget_tool_response()` or `register_decorated_widgets()` - it all happens automatically!
100 changes: 100 additions & 0 deletions examples/widget_test/pizzaz_server.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
"""Pizzaz Widget Demo Server

This demonstrates FastMCP's OpenAI widget support using the Pizzaz mapping library.
The widget renders an interactive map when you ask about pizza toppings.

Example usage:
Show me a pizza map for pepperoni
Create a pizza map with mushrooms
"""

from fastmcp import FastMCP

app = FastMCP("pizzaz-demo")

# Pizzaz widget HTML with the library loaded from OpenAI's CDN
PIZZAZ_HTML = """
<div id="pizzaz-root"></div>
<link rel="stylesheet" href="https://persistent.oaistatic.com/ecosystem-built-assets/pizzaz-0038.css">
<script type="module" src="https://persistent.oaistatic.com/ecosystem-built-assets/pizzaz-0038.js"></script>
"""


@app.ui.openai.widget(
name="pizza-map",
template_uri="ui://widget/pizza-map.html",
html=PIZZAZ_HTML,
title="Pizza Map",
description="Show an interactive pizza map for a given topping",
invoking="Hand-tossing a map",
invoked="Served a fresh map",
widget_csp_resources=["https://persistent.oaistatic.com"],
)
def show_pizza_map(topping: str) -> dict:
"""Show an interactive pizza map for the given topping.

Args:
topping: The pizza topping to map (e.g., "pepperoni", "mushrooms")

Returns:
Structured data for the widget to render
"""
return {
"pizza_topping": topping.strip(),
"map_type": "delicious",
}


@app.ui.openai.widget(
name="pizza-tracker",
template_uri="ui://widget/pizza-tracker.html",
html=PIZZAZ_HTML,
title="Pizza Tracker",
description="Track a pizza order with real-time updates",
invoking="Tracking your pizza",
invoked="Pizza located!",
widget_csp_resources=["https://persistent.oaistatic.com"],
)
def track_pizza(order_id: str) -> tuple[str, dict]:
"""Track a pizza order by order ID.

This demonstrates returning both narrative text and structured data.

Args:
order_id: The pizza order ID to track

Returns:
Tuple of (narrative text, structured data for widget)
"""
narrative = f"Tracking pizza order {order_id}. Your pizza is on the way!"
data = {
"order_id": order_id,
"status": "out_for_delivery",
"estimated_time": "15 minutes",
"driver_location": {"lat": 47.6062, "lng": -122.3321},
}
return narrative, data


@app.ui.openai.widget(
name="pizza-status",
template_uri="ui://widget/pizza-status.html",
html=PIZZAZ_HTML,
title="Pizza Status",
description="Get the current status of your pizza",
widget_csp_resources=["https://persistent.oaistatic.com"],
)
def pizza_status() -> str:
"""Get a simple status message about pizza availability.

This demonstrates returning text only (no structured data).

Returns:
Status message text
"""
return "Pizza ovens are hot and ready! 🍕 We can make any topping you'd like."


if __name__ == "__main__":
# Run with HTTP transport for testing with ChatGPT
app.run(transport="http", host="0.0.0.0", port=8080)
10 changes: 7 additions & 3 deletions src/fastmcp/server/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -429,9 +429,13 @@ async def sample(
]
elif isinstance(messages, Sequence):
sampling_messages = [
SamplingMessage(content=TextContent(text=m, type="text"), role="user")
if isinstance(m, str)
else m
(
SamplingMessage(
content=TextContent(text=m, type="text"), role="user"
)
if isinstance(m, str)
else m
)
for m in messages
]

Expand Down
9 changes: 9 additions & 0 deletions src/fastmcp/server/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@
from fastmcp.tools import ToolManager
from fastmcp.tools.tool import FunctionTool, Tool, ToolResult
from fastmcp.tools.tool_transform import ToolTransformConfig
from fastmcp.ui import UIManager
from fastmcp.utilities.cli import log_server_banner
from fastmcp.utilities.components import FastMCPComponent
from fastmcp.utilities.logging import get_logger, temporary_log_level
Expand Down Expand Up @@ -192,6 +193,7 @@ def __init__(
duplicate_behavior=on_duplicate_prompts,
mask_error_details=mask_error_details,
)
self._ui_manager: UIManager | None = None
self._tool_serializer = tool_serializer

self._lifespan: LifespanCallable[LifespanResultT] = lifespan or default_lifespan
Expand Down Expand Up @@ -357,6 +359,13 @@ def icons(self) -> list[mcp.types.Icon]:
else:
return list(self._mcp_server.icons)

@property
def ui(self) -> UIManager:
"""Access UI components and integrations."""
if self._ui_manager is None:
self._ui_manager = UIManager(self)
return self._ui_manager

@asynccontextmanager
async def _lifespan_manager(self) -> AsyncIterator[None]:
if self._lifespan_result_set:
Expand Down
5 changes: 5 additions & 0 deletions src/fastmcp/ui/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
"""UI components and integrations."""

from fastmcp.ui.manager import UIManager

__all__ = ["UIManager"]
26 changes: 26 additions & 0 deletions src/fastmcp/ui/manager.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
"""UI manager for FastMCP."""

from __future__ import annotations

from typing import TYPE_CHECKING, Any

if TYPE_CHECKING:
from fastmcp.server.server import FastMCP
from fastmcp.ui.openai import OpenAIUIManager


class UIManager:
"""Manager for UI-related components and integrations."""

def __init__(self, fastmcp: FastMCP[Any]) -> None:
self._fastmcp = fastmcp
self._openai: OpenAIUIManager | None = None

@property
def openai(self) -> OpenAIUIManager:
"""Access OpenAI-specific UI components."""
if self._openai is None:
from fastmcp.ui.openai import OpenAIUIManager

self._openai = OpenAIUIManager(self._fastmcp)
return self._openai
15 changes: 15 additions & 0 deletions src/fastmcp/ui/openai/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
"""OpenAI UI components."""

from fastmcp.ui.openai.manager import OpenAIUIManager
from fastmcp.ui.openai.response import (
WidgetToolResponse,
build_widget_tool_response,
transform_widget_response,
)

__all__ = [
"OpenAIUIManager",
"WidgetToolResponse",
"build_widget_tool_response",
"transform_widget_response",
]
Loading
Loading