[Feature Request] Add Ollama Support

[techteam-pro-discount]

Could you please let us use Ollama instead of OpenAI?

[gravelfreeman]

Not "instead" but add support for any local LLM would be great especially since invoices contains personal data.

Some invoices doesn't contain much personal data so those I'd still send them into OpenAi since it's faster than local LLM.

So it would be great to be able to choose the Ai Agent in a dropdown menu in the import process.

In the meantime if you have Acrobat Pro it's fairly quick to use the Redact function to sanitize metadata and redact personal informations from invoices.

[vas3k]

Yeah, Local LLM support is in my plans for a self-hosted version! 100% agree that for many users, the privacy of their invoices will be the first priority

Just let me polish the MVP and research some ideas for agents/plugins, and I will return to this

[gravelfreeman]

Not sure how it works in other countries but it might be unrealistic for the past year tax return to implement versus the amount of time that's going to be required to implement this. If I may suggest to fix as much bugs as possible for the current version is more realistic and newcomers to the app will find it better and might keep it for the upcoming year. At least it's my case, my accountant haven't received the file yet so I'll get her impressions later this week.

[vas3k]

At least it's my case, my accountant haven't received the file yet so I'll get her impressions later this week.

Pls tell me what they'll say!

[gravelfreeman]

At least it's my case, my accountant haven't received the file yet so I'll get her impressions later this week.

Pls tell me what they'll say!

Yes, it was already my intention! I can already see some things but I'll make a final list with her feedback.

[JermaNoiD]

I've selfhosted this but have temporarily mothballed it until local LLMs arrive. Ollama uses an OpenAI compatible API, so hopefully that makes it's implementation fairly trivial. Looking forward to testing it out

[adryserage]

Feature Request: Ollama Support

Status: Feature Request

Analysis

Adding Ollama support would allow users to run TaxHacker with fully local LLM processing.

Implementation Approach

The LLM provider system in ai/providers/llmProvider.ts and lib/llm-providers.ts is already designed for multiple providers. Adding Ollama would involve:

1. Add Ollama to PROVIDERS array in lib/llm-providers.ts:

{
  key: "ollama",
  label: "Ollama (Local)",
  apiKeyName: "ollama_api_url",  // Actually the Ollama server URL
  modelName: "ollama_model_name",
  defaultModelName: "llava",  // or llama3.2-vision for newer models
  suggestedModels: ["llava", "llava:13b", "bakllava"],
}

2. Add Ollama API integration in ai/providers/llmProvider.ts:

   • Ollama API is OpenAI-compatible at http://localhost:11434/v1/
   • Could use the existing OpenAI client with a different base URL

3. Vision model requirement:

   • TaxHacker requires vision capabilities for invoice analysis
   • Ollama models with vision: llava, bakllava, llama3.2-vision

Considerations

• Ollama doesn't use API keys, but needs a server URL
• Performance will depend on local hardware
• Vision models are resource-intensive

This would be great for privacy-conscious users and those with capable local hardware!

[ksd73429udmn]

Any progress on this? Kindly add this key feature.

[adryserage]

@ksd73429udmn still waiting on #62 to be reviewed

[vas3k]

Hey! Sorry, will review soon. I was spammed by AI slop PRs on my projects recently and lost the track of what’s actually important :(

[Kuechenfenster]

any progress, .. in hong kong non of the given llms are accessible without vpn, therefore a local llm or llm option via openrouter would be more than welcome.

[mmplisskin]

Support added on main with 81

It can be configured in the settings U.I.