With mistral model (from library), I become a pick_tool request as response

[fanningert]

LocalAI version:
v3.12.1

Environment, CPU architecture, OS, and Version:

Fedora 34 - Kernel 6.18.12
APU: AMD Halo Stix (Ryzen AI Max+ 395)

Describe the bug

When I use the Model "mistralai_ministral-3-14b-reasoning-2512-multimodal" (YAML from the library + MCP) with MCP, I get the Request for the Tool back.

My question in German:

Suche mit den Tools in englischer Sprache, gibt mir aber das Endergebnis in deutscher Sprache aus. Wie nutzen aktuell S/4HANA 2023 onPremise. Wie lange unterstützt SAP diese Version noch?

Block Thinking:

No tool selected

Response of model

pick_tool{"reasoning": "I need to gather information about the current usage of S/4HANA 2023 on-premise and its support duration from SAP. To do this, I will use the 'search' tool to find relevant sources or documents in English. After collecting the information, I will summarize it in German for the user.", "tool": "search"}

To Reproduce
YAML of model

backend: llama-cpp
context_size: 128000
function:
    disable_no_action: true
    grammar:
        disable: true
        disable_parallel_new_lines: true
        parallel_calls: true
    json_regex_match:
        - (?s)\[TOOL\_CALLS\](.*)
    replace_function_results:
        - key: (?s)^[^{\[]*
          value: ""
        - key: (?s)[^}\]]*$
          value: ""
        - key: (?s)\[TOOL\_CALLS\]
          value: ""
        - key: (?s)\[\/TOOL\_CALLS\]
          value: ""
    return_name_in_function_response: true
mmap: true
mmproj: llama-cpp/mmproj/mmproj-mistralai_Ministral-3-14B-Reasoning-2512-f32.gguf
name: 01_general_mistral
parameters:
    model: llama-cpp/models/mistralai_Ministral-3-14B-Reasoning-2512-Q4_K_M.gguf
    temperature: 0.7
    top_p: 0.95
stopwords:
    - <|im_end|>
    - <dummy32000>
    - </tool_call>
    - <|eot_id|>
    - <|end_of_text|>
    - </s>
    - '[/TOOL_CALLS]'
    - '[/ACTIONS]'
template:
    chat: |
        {{.Input -}}
    chat_message: |-
        {{if eq .RoleName "user" -}}
        [INST] {{.Content }} [/INST]
        {{- else if .FunctionCall -}}
        [TOOL_CALLS] {{toJson .FunctionCall}} [/TOOL_CALLS]
        {{- else if eq .RoleName "tool" -}}
        [TOOL_RESULTS] {{.Content}} [/TOOL_RESULTS]
        {{- else -}}
        {{ .Content -}}
        {{ end -}}
    completion: |
        {{.Input}}
    function: '[AVAILABLE_TOOLS] [{{range .Functions}}{"type": "function", "function": {"name": "{{.Name}}", "description": "{{.Description}}", "parameters": {{toJson .Parameters}} }}{{end}} ] [/AVAILABLE_TOOLS]{{.Input }}'
    join_chat_messages_by_character: ""
mcp:
  remote: |
    {
      "mcpServers": {
        "sap-docs": {
          "url": "http://127.0.0.1:3122/mcp",
          "token": ""
        }
      }
    }
agent:
  max_attempts: 3              # Maximum number of tool execution attempts (default: 3). Higher values provide more resilience but may increase response time.
  max_iterations: 3            # Maximum number of reasoning iterations (default: 3). More iterations allow for complex multi-step problem solving.
  loop_detection: 3            # Loop detection sensitivity level (default: 0, disabled). Set to a positive integer (e.g., 3) to enable loop detection and prevent infinite execution cycles.
  max_adjustment_attempts: 5   # Maximum adjustment attempts for tool calls (default: 5). Prevents infinite loops when adjusting tool call parameters.
  enable_reasoning: true       # Enable tool reasoning capabilities (default: false). When enabled, the agent uses advanced reasoning to better understand tool results.
  enable_planning: false       # Enable auto-planning capabilities (default: false). When enabled, breaks down complex tasks into manageable steps.
  disable_sink_state: false    # Disable sink state behavior (default: false). When enabled, prevents the agent from entering a sink state.
  force_reasoning_tool: false  # Force reasoning tool usage (default: false). When enabled, always use the reasoning tool in the agent’s reasoning process.
  enable_mcp_prompts: false    # Enable MCP prompts (default: false). When enabled, uses specialized prompts exposed by MCP servers.
  enable_plan_re_evaluator: false # Enable plan re-evaluation (default: false). When enabled, dynamically adjusts execution plans based on results.

Expected behavior
Response for user, not the tool request.

Logs
API-Trace, first request, then response

{
  "model": "01_general_mistral",
  "messages": [
    {
      "role": "user",
      "content": "Suche mit den Tools in englischer Sprache, gibt mir aber das Endergebnis in deutscher Sprache aus. Wie nutzen aktuell S/4HANA 2023 onPremise. Wie lange unterstützt SAP diese Version noch?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "pick_tool",
        "parameters": {
          "type": "object",
          "description": "Pick the most appropriate tool to use based on the reasoning. Choose 'reply' if no tool is needed.",
          "properties": {
            "reasoning": {
              "type": "string",
              "description": "The reasoning for the tool choice"
            },
            "tool": {
              "type": "string",
              "description": "The tool to use",
              "enum": [
                "search",
                "fetch",
                "abap_feature_matrix",
                "reply",
                "reply"
              ]
            }
          },
          "required": [
            "tool"
          ]
        }
      }
    }
  ],
  "tool_choice": {
    "type": "function",
    "function": {
      "name": "pick_tool"
    }
  }
}

{
  "created": 1771757816,
  "object": "chat.completion",
  "id": "9b9b5209-80fb-4562-9c93-7ca7ad2c7a32",
  "model": "01_general_mistral",
  "choices": [
    {
      "index": 0,
      "finish_reason": "stop",
      "message": {
        "role": "assistant",
        "content": "pick_tool{\"reasoning\": \"I need to gather information about the usage and support duration of S/4HANA 2023 on-premise. Since the question is technical and related to SAP, I should use the 'search' tool to find relevant details. The output will be in English, but I will translate the final result into German.\", \"tool\": \"search\"}"
      }
    }
  ],
  "usage": {
    "prompt_tokens": 167,
    "completion_tokens": 83,
    "total_tokens": 250
  }
}

Additional context
No additional context added.