The `Agent.system_prompt` is discarded when `message_history` contain `ModelResponse`

[oscar-broman]

Initial Checks

• I confirm that I'm using the latest version of Pydantic AI

Description

The system prompt is seemingly ignored when commenting out the line mentioned in the MRE below.

Example Code

from openai import AsyncOpenAI
from pydantic_ai import Agent
from pydantic_ai.messages import TextPart, ModelResponse
from pydantic_ai.models.openai import OpenAIModel

async def test_system_prompt_issue() -> None:
    openai_key = ""

    model = OpenAIModel(model_name="gpt-4o-mini", openai_client=AsyncOpenAI(api_key=openai_key))

    agent = Agent(
        model=model,
        system_prompt="Respond with the user's last message in upper-case",
    )

    messages = [
        # If this is commented out, the system prompt will not be overridden and the test will pass
        ModelResponse(parts=[TextPart(content="Hi, write something and I'll repeat it")]),
    ]

    result = await agent.run(
        user_prompt="Test",
        message_history=messages,
        result_type=str,
    )

    assert result.all_messages()[-1].parts[0].content == "TEST"

Python, Pydantic AI & LLM client version

Python 3.12.9
pydantic-ai 0.0.30
openai 1.65.2

[jkuehn]

Hi Oscar, if there is any message_history passed, then the system prompt is not applied. I have it in another scenario: #1023

You can use dynamic system prompts to make sure it is always applied:
https://ai.pydantic.dev/agents/#system-prompts

See also:
#531
#560

[oscar-broman]

Hi @jkuehn, appreciate the info! Although I did try what you suggested and the issue seems to persist.

I think perhaps this specific issue is related to having a ModelResponse in message_history.

Another MRE using dynamic system prompts:

from openai import AsyncOpenAI
from pydantic_ai import Agent
from pydantic_ai.messages import TextPart, ModelResponse
from pydantic_ai.models.openai import OpenAIModel


async def test_system_prompt_issue() -> None:
    openai_key = ""

    model = OpenAIModel(model_name="gpt-4o-mini", openai_client=AsyncOpenAI(api_key=openai_key))

    agent = Agent(
        model=model,
    )

    @agent.system_prompt
    def sys_prompt() -> str:
        return "Respond with the user's last message in upper-case"

    messages = [
        # If this is commented out, the system prompt will not be overridden and the test will pass
        ModelResponse(parts=[TextPart(content="Hi, write something and I'll repeat it")]),
    ]

    result = await agent.run(
        user_prompt="Test",
        message_history=messages,
        result_type=str,
    )

    assert result.all_messages()[-1].parts[0].content == "TEST"

[jkuehn]

Hi @oscar-broman, you're right, it does not work. Also with @agent.system_prompt(dynamic=True) not as it reevaluates already existing system messages.

We tested it with a patch for #1023 and your use case works too. It is independent of the type of the message. If there is message history, system prompt is not checked anymore.

[jkuehn]

@oscar-broman if you haven't solved it yet, you can use this decorator to overwrite the default behaviour for now. Just run it somewhere once before initializing your Agent:

def prepare_messages_decorator(original_func):
    async def wrapper(self, user_prompt, message_history, run_context):
        history, next_message = await original_func(self, user_prompt, message_history, run_context)

        # Check if the first message is a system message.
        has_system_message = False
        if history:
            first_message = history[0]
            if isinstance(first_message, _messages.ModelRequest) and any(
                    isinstance(part, _messages.SystemPromptPart) for part in first_message.parts
            ):
                has_system_message = True
        elif next_message and next_message.parts:
            if isinstance(next_message.parts[0], _messages.SystemPromptPart):
                has_system_message = True

        if not has_system_message:
            # Call to self._sys_parts(run_context) is done on the correct instance.
            parts = await self._sys_parts(run_context)
            system_request = _messages.ModelRequest(parts)

            if history:
                # If history is not empty, prepend the system request.
                history = [system_request] + history
            else:
                # If history is empty, prepend the system parts to the next_message.
                if next_message is not None and next_message.parts:
                    # Create a new request consisting of system parts followed by the existing parts.
                    next_message = _messages.ModelRequest(system_request.parts + next_message.parts)
                else:
                    # If next_message is None or has no parts, use the system_request.
                    next_message = system_request

        return history, next_message

    return wrapper


# Applying the decorator to the method
UserPromptNode._prepare_messages = prepare_messages_decorator(UserPromptNode._prepare_messages)

[dinhngoc267]

@oscar-broman if you haven't solved it yet, you can use this decorator to overwrite the default behaviour for now. Just run it somewhere once before initializing your Agent:

def prepare_messages_decorator(original_func):
    async def wrapper(self, user_prompt, message_history, run_context):
        history, next_message = await original_func(self, user_prompt, message_history, run_context)

        # Check if the first message is a system message.
        has_system_message = False
        if history:
            first_message = history[0]
            if isinstance(first_message, _messages.ModelRequest) and any(
                    isinstance(part, _messages.SystemPromptPart) for part in first_message.parts
            ):
                has_system_message = True
        elif next_message and next_message.parts:
            if isinstance(next_message.parts[0], _messages.SystemPromptPart):
                has_system_message = True

        if not has_system_message:
            # Call to self._sys_parts(run_context) is done on the correct instance.
            parts = await self._sys_parts(run_context)
            system_request = _messages.ModelRequest(parts)

            if history:
                # If history is not empty, prepend the system request.
                history = [system_request] + history
            else:
                # If history is empty, prepend the system parts to the next_message.
                if next_message is not None and next_message.parts:
                    # Create a new request consisting of system parts followed by the existing parts.
                    next_message = _messages.ModelRequest(system_request.parts + next_message.parts)
                else:
                    # If next_message is None or has no parts, use the system_request.
                    next_message = system_request

        return history, next_message

    return wrapper


# Applying the decorator to the method
UserPromptNode._prepare_messages = prepare_messages_decorator(UserPromptNode._prepare_messages)

Hello @jkuehn ,

Did you check the approach that set dynamic = True in the decorator get_system_prompt? Does it solve this one. As I tried, and it worked but I noticed the system prompt didn't change (first message in the list) . But set to False make it not work. I try to investigate to give an explanation but couldn't. (I use graph agent)

[jkuehn]

Hi @dinhngoc267, as I wrote before, it depends if you already have a system message or not. If you have no system message, then dynamic = True has no effect. Also in the recent version 0.0.52.

What you have sounds like a different issue. If you have a system message, then it should be reevaluated with dynamic = True. I would expect that.

[dinhngoc267]

Hi @dinhngoc267, as I wrote before, it depends if you already have a system message or not. If you have no system message, then dynamic = True has no effect. Also in the recent version 0.0.52.

What you have sounds like a different issue. If you have a system message, then it should be reevaluated with dynamic = True. I would expect that.

sorry what do you mean by reevaluated, as I remember the last time I checked, dynamic=True only used to change the placeholder in the prompt (for example, changed via the value in the dependencies). But I did do a test, it clearly used the system prompt of another agent (even the history messages was used) . I just don't know how to track this. because when I print out the history message, it doesn't show the system prompt which the system use for that agent (maybe it just run underground or something)

[Kludex]

I'm working on #1360. Can people check to see if it solves the issue here?