allow to use locally installed OLLAMA

[pcause]

I alrreayd have phi4 installed with ollama. Can you allow this to use ollma installed models? If you do, also support the OLLAMA_HOST spec to find the ollama instance

[codingmoh]

Alright! - Sure, I'll have a look at it!

[ivostoykov]

+1

It makes no sense to download again. Please add some settings somewhere - ie. ~/.open-codex where the model can be changed. Perhaps such file may be useful for additional tuning

[rpCal]

+1

[codingmoh]

Okay, an agent using chatcompletions API is required for that, which can be achieved now, especially if we only stick to bash completions for the moment.

I'll draft some changes tomorrow, maybe done by Wednesday. Thanks

[ivostoykov]

Just another hint ;-) if using existing api will be inappropriate, perhaps just setting the path to a gguf file will be enough. But this means to handle various LLMs

[codingmoh]

@ivostoykov Thanks for the suggestion! It’s like we had the same thought at the exact same time – I just checked where ollama stores the models to avoid multiple downloads!

They seem to be stored here on macOS: ~/.ollama/models

My overall goal was actually to optimize prompts and agents (code implementations) for a specific model, because each model requires a certain level of optimization to ensure the best quality, right?

In the case of ollama:
Right now, I don’t know yet what I’ll stick to, so we’d probably have to provide a general-purpose prompt for ollama models.

One idea could also be to introduce an OPEN_CODEX_PROMPT / OPEN_CODEX_PROMPT_TXT environment variable for models brought by the user - right now, this approach sounds reasonable to me.

[ivostoykov]

@codingmoh, Depending on how Ollama is executed, the location may vary. Allowing users to specify the exact path would enhance convenience. Alternatively, searching for *.gguf files and presenting them for user selection is another viable approach.

cheers

[codingmoh]

Oh, I didn’t know that the path could vary.
Honestly, I think the idea of searching for GGUF files isn’t bad at all, I have to admit.

The question, of course, is how this implementation could look like:
Which folders would you search? For example, would you search the .cache folder, the home directory, or really the entire filesystem?

But honestly, I really like the idea or the general approach.

We could also rely on locations that are "known," like .huggingface, .ollama, and whatever else comes along.

It’s actually not a bad idea at all.

[ivostoykov]

Indeed searching for GGUF is a bit tricky because of the access. My preferences go with the config file where the user could place link to the desired models (GGUFs). At the end of the day this is an app for people having some knowledge of the dir architecture, so it shouldn't be a problem. This seems to me the easiest and fastest approach.

[codingmoh]

I'll set up the ability to lookup gguf via config file in the next version, as I'd have to check how to "mount" effectively. We could introduce a GGUF_DIR env variable, and DEFAULT_MODEL, couldn't we? - Or is there any additional config you're thinking of needing?

For now I have added support for ollama now via ollama api (and its python sdk), the pr is attached to the issue, in case you'd like to have a look at the implementation.

I'll review again more carefully tomorrow, and merge it

Essentially I've added:

• Added --ollama flag to enable Ollama mode
• Added --model flag to specify the model (e.g., llama3)

[ivostoykov]

Yes, this is the right direction. About the additional config, I believe it could be ~/.open-codex/open-codex.conf where all needed variables may be placed (sys instruct for instance) then on start check env variables and open-codex.conf merging the result. This will be flexible enough, I believe.

[codingmoh]

Would you suggest to have .env overwriting open-codex.conf?