chore: Upgrade default model to command-r-08-2024

[vblagoje]

Why:

This PR upgrades the Cohere integration to use the command-r-08-2024 chat model. Compared to command-r, the command-r-08-2024 model seems to have more modern tool support and it works well with Haystack Agent. The old command-r had issues with resolving tools with OpenAI tool JSON format which we use. This description has been updated after we found out that command-r-08-2024 works well with our tooling architecture. The discussion below refers to command-r-plus which we though of using but have since found an update to command-r that is command-r-08-2024 - which works well.

What:

• Updated the Cohere integration to reference command-r-08-2024 instead of command-r.
• Adjusted associated test cases to validate against the updated model.

How can it be used:

No changes needed from the user—new pipelines using Cohere will default to the upgraded model automatically.

How did you test it:

• Ran unit tests to validate output structure and model invocation.
• Confirmed compatibility with existing tool use cases and RAG patterns.

Notes for the reviewer:

• Focus on verifying model name substitution and test coverage.

[vblagoje]

@julian-risch in addition manually verified to work nicely with agents.

[julian-risch]

Given that the costs of command-r-plus are about ten times as high as costs of command-r, I don't think it's a good idea to change the default model. Users might be surprised by the increase of costs, no?

[julian-risch]

Updating only the usage example and the https://haystack.deepset.ai/integrations/cohere integration page seems good to me though.

[vblagoje]

Ok, I don't mind, but command-r is two generations behind and command-r-plus cost is inline with top of the line models from various providers

[vblagoje]

We can leave this PR around and trigger integration - when needed

[julian-risch]

Let's get an opinion from another team member. We could release a new major version cohere-v5.0.0 with that change.

And we could also briefly explain the change in the weekly announcement https://github.com/deepset-ai/devrel-board/issues/604

[davidsbatista]

What about a warning, just something in the stdout warning the user that this current release changed the default model, and the price is 10x more? We would then remove it in the next release.

[vblagoje]

Yes, no problem, new piece of information: we should also investigate the situation why command-r doesn't work with our agent while command-r-plus and command-a do. Including multiple tool calls.

Traceback (most recent call last):
  File "/Users/vblagoje/workspace/itinerary-agent/itinerary_agent.py", line 100, in <module>
    main()
  File "/Users/vblagoje/workspace/itinerary-agent/itinerary_agent.py", line 83, in main
    result = agent.run(
  File "/Users/vblagoje/miniconda3/envs/mcp_test/lib/python3.10/site-packages/haystack/components/agents/agent.py", line 255, in run
    llm_messages = Pipeline._run_component(
  File "/Users/vblagoje/miniconda3/envs/mcp_test/lib/python3.10/site-packages/haystack/core/pipeline/pipeline.py", line 62, in _run_component
    raise PipelineRuntimeError.from_exception(component_name, instance.__class__, error) from error
haystack.core.errors.PipelineRuntimeError: The following component failed to run:
Component name: 'chat_generator'
Component type: 'CohereChatGenerator'
Error: status_code: 422, body: {'message': 'No valid tool call or response generated'}

This one I found confusing because we do test for tool calling in cohere with command-r using actual live model yet agent doesn't like this format. Could be something simple, some setting, I'll open a new issue. Here it is: #1693

[vblagoje]

What about a warning, just something in the stdout warning the user that this current release changed the default model, and the price is 10x more? We would then remove it in the next release.

It is very common that such old models are cheap and new ones are more expensive. See the price chat above - nothing out of the ordinary here requiring warnings IMHO

[davidsbatista]

Even if all other models have the same base price as the new default, we should make clear to the user that this integration has changed, and the default model cost is now 10x the previous default.

[davidsbatista]

Maybe a release note is enough, but 10x is a big difference!

[vblagoje]

@julian-risch and @davidsbatista I've updated description to reflect the finding that the command-r-08-2024 works well with out tooling architecture. And it is most likely that the old model, command-r, although compatible with V2 API, isn't actually updated to handle the new tooling format - I could not get it to work with multiple tool definitions in a new OpenAI tool definition format.

[julian-risch]

Looks good to me! 👍
command-r is an alias for command-r-03-2024 (Cohere docs)
and while command-r-03-2024 is not deprecated yet, support for models fine-tuned with Command-R-03-2024 ended on March 08, 2025. (Cohere docs)
For that reason, I believe it's a good idea to update our default from command-r-03-2024 to command-r-08-2024. And my understanding is that the price doesn't change at all, which is good.

[vblagoje]

Included in https://pypi.org/project/cohere-haystack/4.1.0/ release