JSON response format failing to retrieve clean JSON

[serferdinand2]

Issues

When asking the model to return JSON via responseFormat (or response_format) setting in the request options, sometimes with JSON an additional text is added, usually explaining the JSON that it returned.

Reproduction:

String userPrompt is generated dynamically based on the usecase.

const userPrompt = `Given the following Typescript Types: 'interface Question<T extends QuestionType> { title: string; type: T; 

.....  
, your job is to provide structured JSON formatted questions in an array. Create questions for a rating type question survey. Do not include any text in your response except JSON. Do not return Markdown or HTML. Be sure to create not more than 1 questions. Be sure to create at least 1 questions.`


mistral = new MistralClient(options.apiKey);
const chat = await mistral.chat({
		model: 'mistral-small,
		messages: [{ role: 'user', content: userPrompt }],
		responseFormat: { type: 'json_object' },
		temperature: temperature,
	});
data = JSON.parse(
			chat.choices[0].message.content
		) 
		
console.log(data)

Completion:  {
  id: '0ef53d7c6c3a4a5ebbaf43aa96c0f5d9',
  object: 'chat.completion',
  created: 1713366257,
  model: 'mistral-small',
  choices: [
    {
      index: 0,
      message: {
        role: 'assistant',
        content: '[\n' +
          '{\n' +
          '"title": "How would you rate your overall satisfaction with our product?",\n' +
          '"type": "rating",\n' +
          '"required": true,\n' +
          '"properties": {\n' +
          '"labels": [\n' +
          '{\n' +
          '"label": "Very unsatisfied",\n' +
          '"value": 1\n' +
          '},\n' +
          '{\n' +
          '"label": "Unsatisfied",\n' +
          '"value": 2\n' +
          '},\n' +
          '{\n' +
          '"label": "Neutral",\n' +
          '"value": 3\n' +
          '},\n' +
          '{\n' +
          '"label": "Satisfied",\n' +
          '"value": 4\n' +
          '},\n' +
          '{\n' +
          '"label": "Very satisfied",\n' +
          '"value": 5\n' +
          '}\n' +
          ']\n' +
          '}\n' +
          '}\n' +
          ']\n' +
          '\n' +
          **'This JSON array contains one question of type "rating" that asks the customer about their overall satisfaction with the product. The question has 5 labels with corresponding values from 1 to 5, where 1 means "Very unsatisfied" and 5 means "Very satisfied". The question is required.',**
        tool_calls: null
      },
      finish_reason: 'stop',
      logprobs: null
    }
  ],
  usage: { prompt_tokens: 447, total_tokens: 658, completion_tokens: 211 }
}

Notes

The request header seems to be doing fine:

request {
  model: 'mistral-small',
  messages: [
    {
      role: 'user',
      content: 'Given the following Typescript Types \

.......


Do not return Markdown or HTML. Be sure to create not more than 1 questions. Be sure to create at least 1 questions.'
    }
  ],
  tools: undefined,
  temperature: 0,
  max_tokens: undefined,
  top_p: undefined,
  random_seed: undefined,
  stream: false,
  safe_prompt: undefined,
  tool_choice: undefined,
  response_format: { type: 'json_object' }
}

[draqos]

Hello, i am facing similar issues. Did you get a solution ?
Related question: any ideeas on how to enforce the structure of the output JSON ?

[fbelemcatho]

I usually perform a post-processing step ignoring everything before the first { and after the last } of the output text

[fbelemcatho]

However, I have observed many cases of ill-generated JSONs since July/2014 (model mixtral-8x7b), such as:
{",": {",
": {",