ollama support?

[txhno]

Is your feature request related to a problem? Please describe.
would want to reuse the models that I already have downloaded on ollama

Describe the solution you'd like
being able to use models.ollama(model_name_or_path)

Describe alternatives you've considered
llama cpp works as of now, but ollama would just make process of using this app a lot more user friendly having downloads automated and models stored centrally

Additional context
none

[Harsha-Nori]

@txhno Sorry about that random weird comment...removed your reply too since it had a quote of the link in it, hope that's OK!

On topic -- exploring Ollama support is a really good idea. My understanding is that they just use llama.cpp under the hood and manage GGUF files, right? If we can figure out where the GGUFs are hosted on local file systems, then we can use our llama.cpp infrastructure to make it easy to load ollama models in guidance.

We can put this on our backlog to investigate, but if you (or anyone reading this!) have some knowledge about how Ollama works, I'd be happy to tag-team and support a PR here.

@riedgar-ms @nking-1 for awareness

[microdev1]

Hi! I’ve implemented a thin wrapper for Ollama support in my fork. Can you give it a shot before I submit a PR? Thanks!