gpt-4.1-mini ignores valid ResponseFormat schema on follow-up messages and returns malformed JSON containing combined responses

[SaurabhDesaiCeloxis]

Hello Spring AI team,

I am using Spring AI’s ResponseFormat with a JSON schema to get structured outputs from the model. The response_format is correctly provided for every request, and on the first message, both gpt-4o-mini and gpt-4.1-mini respond correctly with structured data matching the schema.

However, on follow-up messages, when I include previous user messages by appending them as plain text (concatenated string), gpt-4.1-mini no longer respects the schema. Instead, it returns malformed JSON that seems to combine responses for both the first and the last question within the same string. This output is invalid and does not adhere to the schema at all.

In contrast, gpt-4o-mini handles this scenario perfectly, returning correct structured responses even with the same concatenation of previous context. Also, this approach works well when using LangChain4j with gpt-4.1-mini.

Why would gpt-4.1-mini ignore the schema despite it being valid and unchanged, and return such malformed combined JSON output on follow-up prompts?

I am passing the same ResponseFormat with the identical schema for every request. The only difference is how the message content is constructed on follow-ups by including previous messages as plain text. It appears that gpt-4.1-mini is more sensitive or brittle to this approach.

Would appreciate any insights or suggestions.

Thanks!