Function calling not working with Ollama

[mustela]

First of all, thank you for such a great library! I’m loving it!

Second, and sorry in advance—I’m new to the LLM world. I’m trying to run the demo with Ollama and the llama3.1 model. While the chat functionality works fine, the tools added via LLMChain.add_tools/2 are never executed. The LLM does not appear to recognize or call any of the registered tools. Im basically doing:

       LLMChain.new!(%{
         llm:
-          ChatOpenAI.new!(%{
-            model: "gpt-4o",
+          ChatOllamaAI.new!(%{
+            model: "llama3.1",
             # don't get creative with answers
             temperature: 0,
             request_timeout: 60_000,

I've also tried changing the prompt to something like the following, but didn't work neither:

 - A weekly workout plan should be detailed and specific.
+- You have access to the following tools: update_current_user and fitness_logs. Use them when needed. If a user asks something requiring their execution, do not answer directly—use the appropriate tool
instead.

Am I missing some setting? Is this because Ollama doesn’t support this feature?

Thanks a lot!

---

Update:

I guess it is not implemented yet based on the code here

[brainlid]

Hi @mustela!

When ChatOllamaAI was first created, Ollama did not support tool calling. Now Ollama does support it (see ollama/ollama#1729).

Some PR work has been done towards adding Ollama tool calling support, but the PR has stalled out. brainlid/langchain#164 The requested changes weren't picked back up. It's not an area that I'm personally working in, so support here depends on contributions.

So, for now, ChatOllamaAI doesn't support tool calling.

[mustela]

@brainlid after reviewing the code that's what I though. Ill try to update that PR. 🤞🏼