How do I add memory to structured chat agent?

[vikasr111]

I have written a simple function to create and run a structured chat agent. In this agent I am trying to implement memory. Here's the code

from langchain import hub
from langchain.agents import create_structured_chat_agent
from langchain.memory import ConversationBufferWindowMemory, ConversationBufferMemory
from langchain_community.chat_models import ChatOpenAI

def run_agent(query):
  try:
    agent_input = format_input(query)
    # Create the language model
    llm = ChatOpenAI(model="gpt-4-1106-preview", temperature=0)

    # Create the list of tools
    tools = [rest_api_tool, draft_email_tool]

    # Get the prompt to use - you can modify this!
    prompt = hub.pull("hwchase17/structured-chat-agent")    
    print(f"Prompt: {prompt}")

    # Create the memory object
    memory = ConversationBufferWindowMemory(
        memory_key='chat_history',
        k=3,
        return_messages=True
    )

    print(f"Memory: {memory}")

    # Construct the JSON agent
    agent = create_structured_chat_agent(llm, tools, prompt)
    # print(f"Agent: {agent}")

    # Create an agent executor by passing in the agent and tools
    agent_executor = AgentExecutor(
        agent=agent, 
        tools=tools, 
        verbose=True, 
        memory=memory,
        return_intermediate_steps=True, 
        handle_parsing_errors=True
    )

    # Run the agent with the input
    agent_response = agent_executor.invoke({"input": query})

    memory_content = memory.buffer
    print(f"Memory content: {memory_content}")

    # Return the agent response and memory contents
    return agent_response
  except Exception as e:
    print(f"Error in executing agent: {e}")
    return None

I am following the documentation on this link but it's so poor and not helpful at all: https://python.langchain.com/docs/modules/agents/agent_types/structured_chat

When I try to return the memory form this function it's giving me the error. How can I troubleshoot it and implement memory properly? How do I continue the next query with the agent where it can use the memory to answer queries?

[dosubot[bot]]

🤖

Hey @vikasr111! Nice to see you back here. Hope all is well on your end. Let's see if we can sort out this memory issue together.

To implement the memory feature in your structured chat agent, you can use the memory_prompts parameter in the create_prompt and from_llm_and_tools methods. This parameter accepts a list of BasePromptTemplate objects that represent the memory of the chat agent. Here's an example:

from langchain_core.prompts.chat import HumanMessagePromptTemplate

# Create memory prompts
memory_prompts = [
    HumanMessagePromptTemplate(content="I remember you told me your name is Bob.")
]

# Create the agent with memory prompts
agent = StructuredChatAgent.from_llm_and_tools(
    llm=my_llm,
    tools=my_tools,
    memory_prompts=memory_prompts,
)

In this example, the agent will have a memory of a human message saying "I remember you told me your name is Bob." This memory can be used to answer subsequent queries.

To troubleshoot the memory feature, you can check the following:

1. Make sure the memory_prompts parameter is a list of BasePromptTemplate objects.
2. Check the content of the memory prompts. The content should be relevant to the queries that the agent will answer.
3. If the agent is not using the memory to answer queries, check the logic of your language model. The model should be designed to use the memory prompts when generating responses.

You can find more details in the StructuredChatAgent class in the LangChain repository.

Regarding the error you're encountering when trying to return the memory from the function, could you please provide the exact error message? This will help in identifying the issue and providing a more accurate solution.

Sources

• libs/langchain/langchain/agents/structured_chat/base.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[vikasr111]

@dosu-bot I solved the memory issue by implementing it as follows:

from langchain import hub
from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain.prompts import SystemMessagePromptTemplate, HumanMessagePromptTemplate
from langchain.agents import AgentExecutor, create_structured_chat_agent
from langchain.memory import ConversationBufferWindowMemory
from langchain_community.chat_models import ChatOpenAI

def run_agent(query):
  try:
    agent_input = format_input(query)
    # Create the language model
    llm = ChatOpenAI(model="gpt-4-1106-preview", temperature=0)

    # Create the list of tools
    tools = [rest_api_tool, draft_email_tool]

    prompt = ChatPromptTemplate.from_messages([
        SystemMessagePromptTemplate.from_template(
            input_variables=['tool_names', 'tools'],
            template=SYSTEM_PROMPT_TEMPLATE
        ),
        MessagesPlaceholder(variable_name='chat_history', optional=True),
        HumanMessagePromptTemplate.from_template(
            input_variables=["input", "chat_history", "agent_scratchpad"],
            template=HUMAN_PROMPT_TEMPLATE
        )
    ])

    print(f"Prompt: {prompt}")

    # Create the memory object
    memory = ConversationBufferWindowMemory(
        memory_key='chat_history',
        k=3,
        return_messages=True
    )

    # Construct the JSON agent
    agent = create_structured_chat_agent(llm, tools, prompt)

    # Create an agent executor by passing in the agent and tools
    agent_executor = AgentExecutor.from_agent_and_tools(
         agent=agent, 
         tools=tools, 
         verbose=True, 
         memory=memory,
         handle_parsing_errors=True,
         return_intermediate_steps=True, 
    )

    # Run the agent with the input
    agent_response = agent_executor.invoke({
        "input": query
        })

    memory_content = memory.buffer
    print(f"Memory content: {memory_content}")

    # Return the agent response and memory contents
    return agent_response
  except Exception as e:
    print(f"Error in executing agent: {e}")
    return None

The memory is working fine. If you notice I have implemented custom prompt containing the system message and human message prompt template suitable for structured chat agent. I want to add some additional variables to the system prompt. For example I want to add a variable named {rules} and pass the value of rules. How can I do it and pass it's value?

[dosubot[bot]]

🤖

To include an additional variable named {rules} in the system prompt and pass its value using the LangChain Python framework, you need to modify your SystemMessagePromptTemplate initialization. Here's how you can do it:

prompt = ChatPromptTemplate.from_messages([
    SystemMessagePromptTemplate.from_template(
        input_variables=['tool_names', 'tools', 'rules'],
        template=SYSTEM_PROMPT_TEMPLATE
    ),
    MessagesPlaceholder(variable_name='chat_history', optional=True),
    HumanMessagePromptTemplate.from_template(
        input_variables=["input", "chat_history", "agent_scratchpad"],
        template=HUMAN_PROMPT_TEMPLATE
    )
])

In the above code, I added 'rules' to the input_variables list of the SystemMessagePromptTemplate. Now, you can include {rules} in your SYSTEM_PROMPT_TEMPLATE string, and the value of rules will be replaced when the template is formatted.

When you format the chat prompt, make sure to pass the rules value. For example:

formatted_prompt = prompt.format_prompt(
    tool_names=tool_names,
    tools=tools,
    rules=rules,
    chat_history=chat_history,
    input=input,
    agent_scratchpad=agent_scratchpad
)

In the above code, rules is a new variable that you need to define before formatting the prompt.

Please note that all placeholders in your template string should have corresponding keys in **kwargs when calling format_prompt. If a key is missing, a KeyError will be raised.

For more information, you can refer to the LangChain core library and the ChatPromptTemplate.from_messages method.

Sources

• libs/core/langchain_core/prompts/string.py
• libs/core/langchain_core/prompts/chat.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[vikasr111]

@dosu-bot How does it fit into my current implementation? Should I pass formatted_prompt while creating structured chat agent? Give me complete solution.

# Construct the JSON agent
agent = create_structured_chat_agent(llm, tools, prompt)

# Create an agent executor by passing in the agent and tools
agent_executor = AgentExecutor.from_agent_and_tools(
     agent=agent, 
     tools=tools, 
     verbose=True, 
     memory=memory,
     handle_parsing_errors=True,
     return_intermediate_steps=True, 
)

# Run the agent with the input
agent_response = agent_executor.invoke({"input": query})

[dosubot[bot]]

🤖

To include a new {rules} variable in the system prompt and pass its value when creating a structured chat agent, you can modify your code as follows:

First, add the {rules} variable to your system prompt template:

SYSTEM_PROMPT_TEMPLATE = """
System: You are using {tool_names}. {rules}
"""

prompt = ChatPromptTemplate.from_messages([
    SystemMessagePromptTemplate.from_template(
        input_variables=['tool_names', 'tools', 'rules'],
        template=SYSTEM_PROMPT_TEMPLATE
    ),
    MessagesPlaceholder(variable_name='chat_history', optional=True),
    HumanMessagePromptTemplate.from_template(
        input_variables=["input", "chat_history", "agent_scratchpad"],
        template=HUMAN_PROMPT_TEMPLATE
    )
])

Then, when you create the structured chat agent, pass the rules value:

rules = "Your custom rules here."

# Construct the JSON agent
agent = create_structured_chat_agent(llm, tools, prompt, rules)

Finally, modify the create_structured_chat_agent function to accept the rules argument and include it in the prompt:

def create_structured_chat_agent(
    llm: BaseLanguageModel, tools: Sequence[BaseTool], prompt: ChatPromptTemplate, rules: str
) -> Runnable:
    missing_vars = {"tools", "tool_names", "agent_scratchpad", "rules"}.difference(
        prompt.input_variables
    )
    if missing_vars:
        raise ValueError(f"Prompt missing required variables: {missing_vars}")

    prompt = prompt.partial(
        tools=render_text_description(list(tools)),
        tool_names=", ".join([t.name for t in tools]),
        rules=rules,
    )
    llm_with_stop = llm.bind(stop=["\nObservation"])
    agent = (
        RunnablePassthrough.assign(
            agent_scratchpad=lambda x: format_log_to_messages(
                x["intermediate_steps"], template_tool_response=TEMPLATE_TOOL_RESPONSE
            )
        )
        | prompt
        | llm_with_stop
        | JSONAgentOutputParser()
    )
    return agent

In this modification, I added rules as an argument to the create_structured_chat_agent function and included it in the missing_vars set to ensure it's provided in the prompt's input variables. Then, I passed rules to the prompt.partial method to include it in the system prompt.

Please note that you need to define the rules variable with your custom rules before passing it to the create_structured_chat_agent function.

This solution is based on the LangChain's json_chat/base.py and chat.py files.

Sources

• libs/langchain/langchain/agents/json_chat/base.py
• libs/core/langchain_core/prompts/chat.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[syntaxtrash]

You can use RunnableWithMessageHistory with InMemoryChatMessageHistory.

For example:

from langchain.agents import create_structured_chat_agent, AgentExecutor
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_core.chat_history import InMemoryChatMessageHistory
from langchain_google_genai import ChatGoogleGenerativeAI
from langchain import hub

from dotenv import load_dotenv
load_dotenv()


def get_message_history(session_id) -> InMemoryChatMessageHistory:
    if session_id not in store:
        store[session_id] = InMemoryChatMessageHistory()
    
    return store[session_id]

store = {} # should be a database
llm = ChatGoogleGenerativeAI(model="gemini-2.0-flash", temperature=0.0)
prompt = hub.pull("hwchase17/structured-chat-agent")

structured_chat_agent = create_structured_chat_agent(
    llm=llm, 
    tools=[], 
    prompt=prompt
)

structured_chat_agent_executor = AgentExecutor(
    agent=structured_chat_agent, 
    tools=[], 
    verbose=True
)

structured_chat_agent_executor_with_history = RunnableWithMessageHistory(
    structured_chat_agent_executor,
    get_message_history,
    input_messages_key="input",
    history_messages_key="chat_history"
)

config = {
    "configurable": {
        "session_id": 1
    }
}

structured_chat_agent_executor_with_history.invoke({"input": "I am aaron."}, config=config)
structured_chat_agent_executor_with_history.invoke({"input": "what is my name?"}, config=config)
print(store)

Reference: https://python.langchain.com/docs/how_to/migrate_agent/#in-langchain