RunnableWithMessageHistory does not work with ChatAnthropic - AsyncRootListenersTracer.on_chain_end error

[selvaradov]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

Try the following code:

from langchain_openai import OpenAIEmbeddings
from langchain_anthropic import ChatAnthropic
from langchain.agents import create_tool_calling_agent, AgentExecutor
from langchain.tools.retriever import create_retriever_tool
from langchain_core.prompts import (
    ChatPromptTemplate,
    MessagesPlaceholder,
)
from langchain.chains.query_constructor.schema import AttributeInfo
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_community.chat_message_histories import SQLChatMessageHistory
from langchain.retrievers import SelfQueryRetriever
from langchain.schema import Document
# import asyncio
from langchain.vectorstores import Chroma


# Initialize LLM
llm = ChatAnthropic(
        model="claude-3-5-sonnet-20240620",
        max_tokens_to_sample=8192,
    )

example_doc = Document("In 2014 something very important happened")
embeddings = OpenAIEmbeddings()
vectorstore = Chroma.from_documents(documents=[example_doc], embedding=embeddings)

def create_self_query_retriever(vectorstore):
    metadata_field_info = [
        AttributeInfo(
            name="date",
            description=f"The year associated with the information.",
            type="string",
        ),
    ]

    document_content_description = "Landmark developments in AI."

    return SelfQueryRetriever.from_llm(
        llm,
        vectorstore,
        document_content_description,
        metadata_field_info,
    )

self_query_retriever = create_self_query_retriever(vectorstore)

prompt = ChatPromptTemplate.from_messages(
    [
        MessagesPlaceholder(variable_name="history"),
        ("human", "{input}"),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
    ]
)

tool = create_retriever_tool(
    self_query_retriever,
    "ai_retriever",
    "Searches for information about developments in AI.",
)
tools = [tool]

# Create the agent
agent = create_tool_calling_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

# Add message history
agent_with_history_async = RunnableWithMessageHistory(
    agent_executor,
    lambda session_id: SQLChatMessageHistory(
        session_id=session_id,
        connection="sqlite+aiosqlite:///chats.db",
        async_mode=True,
    ),
    input_messages_key="input",
    history_messages_key="history",
)

async def run_agent_with_updates(agent, query, sid):
    config = {"configurable": {"session_id": sid}}

    async for event in agent.astream_events(
        {"input": query}, 
        config,
        version="v2",
    ):
        kind = event["event"]

        if kind == "on_chat_model_stream":
            content = event["data"]["chunk"].content
            if content:
                print(content, end="", flush=True)


await run_agent_with_updates(agent_with_history_async, "What was the main development in AI in 2014?", "123")

And then ask a question relying on context:

await run_agent_with_updates(agent_with_history_async, "Can you tell me what happened the year afterwards?", "123")

Error Message and Stack Trace (if applicable)

WARNING:langchain_core.callbacks.manager:Error in AsyncRootListenersTracer.on_chain_end callback: AttributeError("'dict' object has no attribute 'type'")

Description

• I'm trying to have chat history with an Anthropic model
• Nothing is getting saved to the chats.db SQLite database
• There seems to be something wrong with how the ChatAnthropic class is implemented, because when I replace it with ChatOpenAI, it works fine.
• See also Chat history not getting logged in Redis when using Anthropic based tool calling agent #25223

System Info

System Information
------------------
> OS:  Linux
> OS Version:  #1 SMP PREEMPT_DYNAMIC Thu Jun 27 21:05:47 UTC 2024
> Python Version:  3.10.12 (main, Jul 29 2024, 16:56:48) [GCC 11.4.0]

Package Information
-------------------
> langchain_core: 0.3.0
> langchain: 0.3.0
> langchain_community: 0.3.0
> langsmith: 0.1.121
> langchain_anthropic: 0.2.0
> langchain_chroma: 0.1.4
> langchain_openai: 0.2.0
> langchain_text_splitters: 0.3.0
> langgraph: 0.2.22

Optional packages not installed
-------------------------------
> langserve

Other Dependencies
------------------
> aiohttp: 3.10.5
> anthropic: 0.34.2
> async-timeout: 4.0.3
> chromadb: 0.5.3
> dataclasses-json: 0.6.7
> defusedxml: 0.7.1
> fastapi: 0.114.2
> httpx: 0.27.2
> jsonpatch: 1.33
> langgraph-checkpoint: 1.0.10
> numpy: 1.26.4
> openai: 1.45.1
> orjson: 3.10.7
> packaging: 24.1
> pydantic: 2.9.1
> pydantic-settings: 2.5.2
> PyYAML: 6.0.2
> requests: 2.32.3
> SQLAlchemy: 2.0.34
> tenacity: 8.5.0
> tiktoken: 0.7.0
> typing-extensions: 4.12.2

[keenborder786]

okay I am trying to replicate this issue and see.

[keenborder786]

@selvaradov can you send the entire traceback please?

[selvaradov]

@keenborder786 Thanks! There is no further stack trace that shows up unfortunately, is there a way that I can find it apart from in the terminal / notebook output?

[selvaradov]

Let me know if you can't replicate it, but the code above was sufficient to cause the issue in a fresh Colab notebook for me. (I think the await syntax may be different if not running in Jupiter?)

[keenborder786]

I was not able to replicate it but can you run the following, see if it works for you (This basically follows the old style of using memory but it might work for your use case):

from langchain_openai import OpenAIEmbeddings
from langchain_anthropic import ChatAnthropic
from langchain.agents import create_tool_calling_agent, AgentExecutor
from langchain.tools.retriever import create_retriever_tool
from langchain_core.prompts import (
    ChatPromptTemplate,
    MessagesPlaceholder,
)
from langchain.chains.query_constructor.schema import AttributeInfo
from langchain_community.chat_message_histories import SQLChatMessageHistory
from langchain.retrievers import SelfQueryRetriever
from langchain.schema import Document
import asyncio
from langchain.vectorstores import Chroma
from langchain.memory import ConversationBufferMemory
# Initialize LLM
llm = ChatAnthropic(
        model="claude-3-5-sonnet-20240620",
        max_tokens_to_sample=8192,
    )
example_doc = Document("In 2014 something very important happened")
embeddings = OpenAIEmbeddings()
vectorstore = Chroma.from_documents(documents=[example_doc], embedding=embeddings)



def create_self_query_retriever(vectorstore):
    metadata_field_info = [
        AttributeInfo(
            name="date",
            description=f"The year associated with the information.",
            type="string",
        ),
    ]

    document_content_description = "Landmark developments in AI."

    return SelfQueryRetriever.from_llm(
        llm,
        vectorstore,
        document_content_description,
        metadata_field_info,
    )

self_query_retriever = create_self_query_retriever(vectorstore)

prompt = ChatPromptTemplate.from_messages(
    [
        MessagesPlaceholder(variable_name="history"),
        ("human", "{input}"),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
    ]
)

tool = create_retriever_tool(
    self_query_retriever,
    "ai_retriever",
    "Searches for information about developments in AI.",
)
tools = [tool]

# Create the agent
memory_history = SQLChatMessageHistory(
        session_id="",
        connection="sqlite:///chats.db",
        async_mode=False,
    )
memory = ConversationBufferMemory(chat_memory = memory_history, 
                                  input_key = "input", memory_key = "history", return_messages = True)
agent = create_tool_calling_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True, memory=memory)


async def run_agent_with_updates(agent, query, sid):
    config = {"configurable": {"session_id": sid}}

    async for event in agent_executor.astream_events(
        {"input": query}, 
        config,
        version="v2",
    ):
        kind = event["event"]

        if kind == "on_chat_model_stream":
            content = event["data"]["chunk"].content
            if content:
                print(content, end="", flush=True)
async def main(session_id:str):
    agent_executor.memory.chat_memory.session_id = session_id
    await run_agent_with_updates(agent_executor, "What was the main development in AI in 2014?", "123")
    

asyncio.run(main('foo'))

[selvaradov]

Here is a notebook to replicate the issue precisely: https://colab.research.google.com/drive/1OIziMD6Bk9YEgVWNFycV7aGWEoWoTU8O?usp=sharing. You have to scroll all the way along to the right of the two output cells and you'll see the WARNING:langchain_core.callbacks.manager:Error in AsyncRootListenersTracer.on_chain_end callback: AttributeError("'dict' object has no attribute 'type'"), and then also from the second reply it's clear that the agent isn't able to recall history.

[Edit: I've added some more context to the notebook to show that this is only a problem which occurs with the Anthropic model, not OpenAI ones.]

I'll try your solution and see if it works for my use case. One difference I can see now is that it's not using the async mode for the SQLChat history, which possibly matters

[selvaradov]

Yep, the solution you came up with is working. The main issue about the AsyncRootListenersTracer isn't fixed but this is a helpful workaround, so thanks!

Two questions about the new code:

• when you're creating the memory_history, why is it that session_id gets set to '', and then manually gets modified through agent_executor.memory.chat_memory.session_id = session_id?
• I tried using an async SQLChatMessageHistory with your code but this didn't work, do you know why this would be?

[keenborder786]

@selvaradov :

• Session ID needs to manually updated in this case, for RunnableWithMessageHistory it is done automatically behind the scenes (same thing is being done).
• Async method for SQLChatMessageHistory are not supported when AgentExecutor is executed directly.