create_sql_agent with ChatMistralAI causes this error:"Expected last role User or Tool (or Assistant with prefix True) for serving but got assistant"

[amarion35]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

This is the minimal code to reproduce the error:

import dotenv
import os
from langchain_community.agent_toolkits import create_sql_agent
from langchain_community.utilities import SQLDatabase
from langchain_mistralai import ChatMistralAI

# Get api key from .env file
dotenv.load_dotenv(".dev.env")
api_key = str(os.getenv("MISTRAL_API_KEY"))

# Create langchain database object
db = SQLDatabase.from_uri("postgresql://root:root@localhost:65432/test")

# Create agent
llm = ChatMistralAI(model_name="mistral-small-latest", api_key=api_key)
agent_executor = create_sql_agent(llm, db=db, agent_type="tool-calling", verbose=True)

agent_executor.invoke("Do any correct query")

This is the payload of the call to the mistral API route /chat/completions:

{
    "messages": [
        {
            "role": "system",
            "content": "You are an agent designed to interact with a SQL database.\\nGiven an input question, create a syntactically correct postgresql query to run, then look at the results of the query and return the answer.\\nUnless the user specifies a specific number of examples they wish to obtain, always limit your query to at most 10 results.\\nYou can order the results by a relevant column to return the most interesting examples in the database.\\nNever query for all the columns from a specific table, only ask for the relevant columns given the question.\\nYou have access to tools for interacting with the database.\\nOnly use the below tools. Only use the information returned by the below tools to construct your final answer.\\nYou MUST double check your query before executing it. If you get an error while executing a query, rewrite the query and try again.\\n\\nDO NOT make any DML statements (INSERT, UPDATE, DELETE, DROP etc.) to the database.\\n\\nIf the question does not seem related to the database, just return \"I don\"t know\" as the answer.\\n"
        },
        {
            "role": "user",
            "content": "Do any correct query"
        },
        {
            "role": "assistant",
            "content": "I should look at the tables in the database to see what I can query.  Then I should query the schema of the most relevant tables."
        }
    ],
    "model": "mistral-small-latest",
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "sql_db_query",
                "description": "Input to this tool is a detailed and correct SQL query, output is a result from the database. If the query is not correct, an error message will be returned. If an error is returned, rewrite the query, check the query, and try again. If you encounter an issue with Unknown column "xxxx" in "field list", use sql_db_schema to query the correct table fields.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "query": {
                            "description": "A detailed and correct SQL query.",
                            "type": "string"
                        }
                    },
                    "required": [
                        "query"
                    ]
                }
            }
        },
        {
            "type": "function",
            "function": {
                "name": "sql_db_schema",
                "description": "Input to this tool is a comma-separated list of tables, output is the schema and sample rows for those tables. Be sure that the tables actually exist by calling sql_db_list_tables first! Example Input: table1, table2, table3",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "table_names": {
                            "description": "A comma-separated list of the table names for which to return the schema. Example input: \"table1, table2, table3\"",
                            "type": "string"
                        }
                    },
                    "required": [
                        "table_names"
                    ]
                }
            }
        },
        {
            "type": "function",
            "function": {
                "name": "sql_db_list_tables",
                "description": "Input is an empty string, output is a comma-separated list of tables in the database.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "tool_input": {
                            "description": "An empty string",
                            "default": "",
                            "type": "string"
                        }
                    }
                }
            }
        },
        {
            "type": "function",
            "function": {
                "name": "sql_db_query_checker",
                "description": "Use this tool to double check if your query is correct before executing it. Always use this tool before executing a query with sql_db_query!",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "query": {
                            "description": "A detailed and SQL query to be checked.",
                            "type": "string"
                        }
                    },
                    "required": [
                        "query"
                    ]
                }
            }
        }
    ],
    "stream": true
}

Error Message and Stack Trace (if applicable)

python test.py


> Entering new SQL Agent Executor chain...
Traceback (most recent call last):
  .../test.py", line 18, in <module>
    agent_executor.invoke("Do any correct query")
  .../lib/python3.11/site-packages/langchain/chains/base.py", line 166, in invoke
    raise e
  .../lib/python3.11/site-packages/langchain/chains/base.py", line 156, in invoke
    self._call(inputs, run_manager=run_manager)
  .../lib/python3.11/site-packages/langchain/agents/agent.py", line 1433, in _call
    next_step_output = self._take_next_step(
                       ^^^^^^^^^^^^^^^^^^^^^
  .../lib/python3.11/site-packages/langchain/agents/agent.py", line 1139, in _take_next_step
    [
  .../lib/python3.11/site-packages/langchain/agents/agent.py", line 1139, in <listcomp>
    [
  .../lib/python3.11/site-packages/langchain/agents/agent.py", line 1167, in _iter_next_step
    output = self.agent.plan(
             ^^^^^^^^^^^^^^^^
  .../lib/python3.11/site-packages/langchain/agents/agent.py", line 515, in plan
    for chunk in self.runnable.stream(inputs, config={"callbacks": callbacks}):
  .../lib/python3.11/site-packages/langchain_core/runnables/base.py", line 2882, in stream
    yield from self.transform(iter([input]), config, **kwargs)
  .../lib/python3.11/site-packages/langchain_core/runnables/base.py", line 2869, in transform
    yield from self._transform_stream_with_config(
  .../lib/python3.11/site-packages/langchain_core/runnables/base.py", line 1867, in _transform_stream_with_config
    chunk: Output = context.run(next, iterator)  # type: ignore
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  .../lib/python3.11/site-packages/langchain_core/runnables/base.py", line 2831, in _transform
    for output in final_pipeline:
  .../lib/python3.11/site-packages/langchain_core/runnables/base.py", line 1163, in transform
    for ichunk in input:
  .../lib/python3.11/site-packages/langchain_core/runnables/base.py", line 4784, in transform
    yield from self.bound.transform(
  .../lib/python3.11/site-packages/langchain_core/runnables/base.py", line 1181, in transform
    yield from self.stream(final, config, **kwargs)
  .../lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 265, in stream
    raise e
  .../lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 245, in stream
    for chunk in self._stream(messages, stop=stop, **kwargs):
  .../lib/python3.11/site-packages/langchain_mistralai/chat_models.py", line 523, in _stream
    for chunk in self.completion_with_retry(
  .../lib/python3.11/site-packages/langchain_mistralai/chat_models.py", line 391, in iter_sse
    _raise_on_error(event_source.response)
  .../lib/python3.11/site-packages/langchain_mistralai/chat_models.py", line 131, in _raise_on_error
    raise httpx.HTTPStatusError(
httpx.HTTPStatusError: Error response 400 while fetching https://api.mistral.ai/v1/chat/completions: {"object":"error","message":"Expected last role User or Tool (or Assistant with prefix True) for serving but got assistant","type":"invalid_request_error","param":null,"code":null}

Description

I'm following this guide to implement a SQL agent with the langchain_community.agent_toolkits.create_sql_agent function but instead of using OpenAI I want to use the Mistral API. When I try to implement this agent with mistral I get the error you can see above. The mistral chat completion api doesn't expect the last message of the chat to be an assistant message unless the prefix feature is enabled. I don't know what is the expected behavior of this agent so I can't tell if it's an agent issue, or a mistral client issue.

System Info

# python -m langchain_core.sys_info

System Information
------------------
> OS:  Linux
> OS Version:  #1 SMP Fri Mar 29 23:14:13 UTC 2024
> Python Version:  3.11.6 (main, Nov 22 2023, 18:29:18) [GCC 9.4.0]

Package Information
-------------------
> langchain_core: 0.2.7
> langchain: 0.2.5
> langchain_community: 0.2.5
> langsmith: 0.1.77
> langchain_experimental: 0.0.60
> langchain_mistralai: 0.1.8
> langchain_text_splitters: 0.2.1
> langserve: 0.2.2

Packages not installed (Not Necessarily a Problem)
--------------------------------------------------
The following packages were not found:

> langgraph

[rudder-sanket-gaikwad]

Found any solution to this?
My understanding was agent_type="tool-calling" is generic and supposed to work with all chat models.
But I am also facing same error.

[dosubot]

Hi, @amarion35. I'm Dosu, and I'm helping the LangChain team manage their backlog. I'm marking this issue as stale.

Issue Summary:

• The issue involves an error with the create_sql_agent function when using ChatMistralAI.
• The error is related to role assignment in the API call to Mistral, persisting even after updating to the latest version.
• @rudder-sanket-gaikwad also experiences this issue, questioning the compatibility of agent_type="tool-calling" with all chat models.

Next Steps:

• Could you confirm if this issue is still relevant with the latest version of LangChain? If so, please comment to keep the discussion open.
• If there is no further activity, the issue will be automatically closed in 7 days.

Thank you for your understanding and contribution!

[jodyhuntatx]

Issue still occurs with langchain=0.3.14

$ python3 -m langchain_core.sys_info

System Information

OS: Linux
OS Version: #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024
Python Version: 3.10.12 (main, Nov 6 2024, 20:22:13) [GCC 11.4.0]

Package Information

langchain_core: 0.3.29
langchain: 0.3.14
langsmith: 0.2.10
langchain_mistralai: 0.2.4
langchain_text_splitters: 0.3.5
langgraph_sdk: 0.1.51

[ketanPatil]

Facing the same issue.
Here is the code:

db = SQLDatabase(engine=engine)
llm = init_chat_model("mistral-large-latest", model_provider="mistralai")
agent_executor = create_sql_agent(llm, db=db, agent_type="tool-calling", top_k=50, verbose=True)
agent_executor.invoke({"input": "Query to the model?"})

Error

File .venv\Lib\site-packages\langchain_mistralai\chat_models.py, line 480, in iter_sse
    _raise_on_error(event_source.response)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File .venv\Lib\site-packages\langchain_mistralai\chat_models.py, line 178, in _raise_on_error
    raise httpx.HTTPStatusError(
    ...<4 lines>...
    )
httpx.HTTPStatusError: Error response 400 while fetching https://api.mistral.ai/v1/chat/completions: {"object":"error","message":"Expected last role User or Tool (or Assistant with prefix True) for serving but got assistant","type":"invalid_request_error","param":null,"code":null}