Add MCP Client Support #39
Conversation
|
It's great to see MCP support for Chatlas. A few thoughts about dependencies and requirements: The |
Sure thing! Out of curiosity, is there something that would require us to keep py3.9 support? I suppose that would be considered a breaking change. |
Posit Open Source tries to support all python versions that are receiving security updates. Py3.9 will end its support in Oct '25. Similarly, given the mcp is a non-critical code path, the mcp requirements shouldn't hinder the core functionality from running on an earlier python versions. |
|
Overall, this is looking great, thank you! I think we'll have some additional suggestions for code changes/improvements, but also excited to see what you come up with for docs. |
| @classmethod | ||
| def from_func( |
There was a problem hiding this comment.
I'd like to have this as a subclass to the Tool class.
I'd also like to restructure the Tool class to be an ABC class. Then ToolFunction, ToolMcp, ToolMcpSse, and ToolMcpStdio would all spawn from Tool.
- Tool
- ToolFunction
- ToolMcp
- ToolMcpSse
- ToolMcpStdio
I'm happy to implement this early next week.
There was a problem hiding this comment.
I am not sure this is necessary at this point? Unless I am missing something, converging into a single tool API seems to be sufficient.
Also, curious what we gain by this abstraction:
- ToolMcp
- ToolMcpSse
- ToolMcpStdio
| url: str, | ||
| include_tools: Optional[list[str]] = None, | ||
| exclude_tools: Optional[list[str]] = None, | ||
| transport_kwargs: dict[str, Any] = {}, |
There was a problem hiding this comment.
Is there any way we can support typing on these arguments?
| transport_kwargs: dict[str, Any] = {}, | |
| kwargs: dict[str, Any] = {}, |
| env: dict[str, str] | None = None, | ||
| include_tools: Optional[list[str]] = None, | ||
| exclude_tools: Optional[list[str]] = None, | ||
| transport_kwargs: dict[str, Any] = {}, |
There was a problem hiding this comment.
Is there any way we can support typing on these arguments?
| transport_kwargs: dict[str, Any] = {}, | |
| kwargs: dict[str, Any] = {}, |
|
Echoing my comment here that was made in ellmer's repo:
Curious for your thoughts @cpsievert @schloerke @wch @gadenbuie |
| from mcp import ( | ||
| ClientSession as MCPClientSession, | ||
| ) | ||
| from mcp.client.sse import sse_client | ||
| from mcp.client.stdio import StdioServerParameters, stdio_client |
There was a problem hiding this comment.
I think these would have to be run time imports given mcp is an optional dependency
Looking at things from purely a Python perspective, I'm OK with the additional overhead. It doesn't seem like we're exposing ourselves to a large/complex API dependency, and I'd be fine making breaking changes if and when Obviously, it'd be a bigger lift for ellmer, so I don't want to speak on it's behalf, but I'll discuss with Hadley/Joe.
I'm somewhat open to this, but it's also currently reaching into private APIs, and even with more public APIs it seems fairly non-trivial to follow what's happening. Maybe with some tweaks to chatlas we could make it simpler, but I also feel like if we're gonna officially document something, it's probably worth going all the way with an official wrapper around
I agree, I'd prefer that we make a decision the R side first before making any official moves here. |
That is fair. Maybe we take a different approach then. Something that consumes the Chat object rather than it being contained within it. Then the MCP code can be siloed into its own submodule and not cause issues if the optional mcp dep is not installed. There is state needing to be maintained for the server connections/comms that are currently living in the Chat object which makes optional imports more problematic. I think instead, we make Chat a lower level concept than the MCP client that the client can consume or even spin up on its own if required. Just a thought. from chatlas import mcp, Chat
chat = Chat()
mcp_client = mcp.Client("server path/url")
mcp_client.register_tools(chat) |
|
Very interested in this feature. Any chance you have a timeline? |
|
@mconflitti-pbc Was able to test out your PR. Seems to be working as intended. Thanks! import os
import sys
import requests
from shiny import App, ui
from pydantic import BaseModel, Field
from chatlas import ChatOpenAI
from dotenv import load_dotenv
load_dotenv()
class GetCurrentTemperature(BaseModel):
"""
Get the current weather given a latitude and longitude.
You MUST provide both latitude and longitude as numbers.
If the user provides a location name (not latitude/longitude),
you MUST first call the 'get_lat_long' tool to obtain the coordinates
then call this tool with those coordinates.
"""
latitude: float = Field(
description="The latitude of the location. Must be a float."
)
longitude: float = Field(
description="The longitude of the location. Must be a float."
)
def get_current_temperature(latitude: float, longitude: float) -> dict:
lat_lng = f"latitude={latitude}&longitude={longitude}"
url = f"https://api.open-meteo.com/v1/forecast?{lat_lng}¤t=temperature_2m,wind_speed_10m&hourly=temperature_2m,relative_humidity_2m,wind_speed_10m"
response = requests.get(url)
json = response.json()
return json["current"]
class GetLatLong(BaseModel):
"""
Use this tool to get the latitude and longitude for a location name provided by the user.
If the user asks for weather and only provides a location name, call this tool first to get coordinates.
"""
location: str = Field(
description="The location name to get latitude and longitude for. Must be a string."
)
def get_lat_long(location: str) -> dict:
url = f"https://geocode.maps.co/search?q='{location}'&api_key={os.getenv('GEO_API_KEY')}"
response = requests.get(url)
json = response.json()
if len(json) == 0:
raise ValueError(
f"Could not find location: {location}. Try to determine the location from the LLM response and call this tool again with the new location."
)
else:
return {"latitude": float(json[0]["lat"]), "longitude": float(json[0]["lon"])}
app_ui = ui.page_fluid(ui.panel_title("Weather Assistant"), ui.chat_ui(id="my_chat"))
def server(input, output, session):
chat = ui.Chat(id="my_chat")
chat_client = ChatOpenAI(
model="gpt-4o",
api_key=os.getenv("OPENAI_API_KEY"),
)
if hasattr(chat_client, "register_mcp_stdio_server_async"):
# register MCP server at session start
# this requires that the development version of chatlas with
# MCP client support is installed using the following command:
# uv add git+https://github.com/vnijs/chatlas-mcp
@session.on_flush
async def setup():
await chat_client.register_mcp_stdio_server_async(
name="weather-mcp",
command=sys.executable,
args=["2e-mcp-llama-server.py"],
)
else:
# if you don't have development version of chatlas with
# MCP client support installed you can use the following to
# register the tools
chat_client.register_tool(get_lat_long, model=GetLatLong)
chat_client.register_tool(get_current_temperature, model=GetCurrentTemperature)
@chat.on_user_submit
async def handle_user_input(user_input: str):
response = await chat_client.stream_async(user_input)
await chat.append_message_stream(response)
app = App(app_ui, server)MCP server: import os
import requests
from mcp.server.fastmcp import FastMCP
from dotenv import load_dotenv
load_dotenv()
mcp = FastMCP("weather-mcp")
@mcp.tool()
def get_current_temperature(latitude: float, longitude: float) -> dict:
"""
Get the current weather given a latitude and longitude.
You MUST provide both latitude and longitude as numbers.
If the user provides a location name (not latitude/longitude),
you MUST first call the 'get_lat_long' tool to obtain the coordinates
then call this tool with those coordinates."
Parameters
----------
latitude : float
The latitude of the location.
longitude : float
The longitude of the location.
"""
lat_lng = f"latitude={latitude}&longitude={longitude}"
url = f"https://api.open-meteo.com/v1/forecast?{lat_lng}¤t=temperature_2m,wind_speed_10m&hourly=temperature_2m,relative_humidity_2m,wind_speed_10m"
response = requests.get(url)
json = response.json()
return json["current"]
@mcp.tool()
def get_lat_long(location: str) -> dict:
"""
Use this tool to get the latitude and longitude for a location name provided by the user.
If the user asks for weather or temperature and only provides a location name, call this tool first to get coordinates.
If the location is not found, raise a ValueError with a message indicating the location was not found and
then try to determine the location from the LLM response and call this tool again with the new location.
Example
-------
>>> get_lat_long("La Jolla, California")
{'latitude': 32.8801, 'longitude': -117.2528}
Parameters
----------
location : str
The location to get the lat and long for.
"""
url = f"https://geocode.maps.co/search?q='{location}'&api_key={os.getenv('GEO_API_KEY')}"
response = requests.get(url)
json = response.json()
if len(json) == 0:
raise ValueError(
f"Could not find location: {location}. Try to determine the location from the LLM response and call this tool again with the new location."
)
else:
return {"latitude": float(json[0]["lat"]), "longitude": float(json[0]["lon"])}
if __name__ == "__main__":
mcp.run(transport="stdio")
# if your venv is setup correctly, you can run this with:
# uv run mcp dev 2e-mcp-llama-server.py |
Issue #21
Adds ability to register an SSE or stdio MCP server with any Chat provider.
Tests provide contained examples of these MCP servers that the tests interact with to register tools remotely.
Slightly modifies how tools are created:
Tool.from_func(maintains existing behavior) andTool.from_mcpstatic methods.The use of MCP does mean that Python 3.9 support would be dropped. If this is a major problem, happy to brainstorm ways to make MCP an extra and conditionally import it when needing it.
TODOs: