Non-local Ollama server Fail to connect

[mando222]

Describe the bug

In the ouput I keep getting the app-dev_1 | WARN Constants Failed to get Ollama models: fetch failed messages over and over. My setup is a bit different from most of the posts I see here as I am running a Ollama server on my local network. The URL is set in the .env.local file and it is being set in the UI from what I can tell because the UI seems to have a value set as you can see in the screenshot below.

This seems similar to #721 but the fix that worked for him is not working for me.

I know the Ollama server is running and ports are open since I also run an openwebui frontend from the same box I am trying to run bolt.diy from. In other words the route is tested every time I run a query in openwebui.

Additionally the model select dropdown is not populated with any options as I would expect if the connection was just full failing.

My architecture is pretty simple. Ollama server on local network-> Web server for Bolt.diy on different box on local network.

Link to the Bolt URL that caused the error

Not public accessable

Steps to reproduce

1. Install bolt.diy on a different box form the Ollama server
2. Configure the URL in the .env.local file
3. start the bolt.diy server
4. navigate to the UI and it should be broken

Expected behavior

I expected the bolt.diy frontend to be able to find the Ollama server

Screen Recording / Screenshot

Platform

• OS: Ubuntu
• Browser: Brave
• Version: bolt@0.0.1 dev

Provider Used

No response

Model Used

No response

Additional context

No response

[thecodacus]

can you try setting the url in the setting tab?

[mando222]

Yeah tried that as well

But the debug still gives

And the terminal output is still

[mando222]

Oh and just to show you my .env.local

[thecodacus]

are these in separate vm or container? can you tell us about your configurations or setup on how these are running (ollama and bolt)?

[k1rb]

Same issue here. I've tried git clone of main as well as download of the v0.0.2 tar.gz.

I've tried setting environment variables OLLAMA_API_BASE_URL=http://192.168.1.10:11434. I've tried .env (also removed this file from .dockerignore). I've also tried setting this value through the UI. No matter how it's set I get this warning in stdout:

[WARN] Failed to get Ollama models:  Network connection lost.

Using...

• Docker host OS: Debian 12.7 amd64
• Docker version 20.10.24+dfsg1, build 297e128
• Docker Compose version v2.29.2
• Ollama 0.4.4 at http://192.168.1.10:11434 (no acls etc and reachable by other containers and network devices)
• Deployment: docker compose --profile production up -d --build with configuration in, .env

From within the bolt container I can successfully curl http://192.168.1.10:11434/api/tags to enumerate ollama models. In the container .env.local is symlinked to .env file so, if the application is looking for either will find the same configuration.

are these in separate vm or container? can you tell us about your configurations or setup on how these are running (ollama and bolt)?

In my case both containers (ollama and bolt) are running from the same docker host with host port binds and no acls.

[thecodacus]

did you try setting the base url from settings settings tab?

[k1rb]

did you try setting the base url from settings settings tab?

Yes. I've tried http://${ipAddress}:${port}, http://${ipAddress}:${port}/api, and same paths via https and reverse proxy. I've also used invalid URL's and it generates more/different errors.

[g7eg10r]

Hi.

Same issue as @k1rb here (running Bolt in a container and Ollama in another one, on the same host).

I managed to have the models list loaded from Ollama by setting the appropriate URL in the web UI, but chat fails.

From what I've seen so far, the server side code never sees the OLLAMA_API_BASE_URL value set in .env file (or web UI level setting), hence it always fallback to the default localhost based URL (this explains why curling Ollama from within Bolt succeeds while still failing when using the app).

Logging the import.meta.env object shows that it is not populated as expected (OLLAMA_API_BASE_URL is an empty string, while there is a sensible value for this variable in .env file, Wrangler even sees it) : {"BASE_URL":"/","DEV":false,"MODE":"production","OLLAMA_API_BASE_URL":"","PROD":true,"SSR":true,"TOGETHER_API_BASE_URL":"","VITE_LOG_LEVEL":"debug"}.

[thecodacus]

Logging the import.meta.env object shows that it is not populated as expected (OLLAMA_API_BASE_URL is an empty string, while there is a sensible value for this variable in .env file, Wrangler even sees it) :

when using the UI to set the baseUrl which I recommend it uses the browser cookies in both server and the frontend to get the configuration so these variables will be ignored

[thecodacus]

Yes. I've tried http://${ipAddress}:${port}, http://${ipAddress}:${port}/api, and same paths via https and reverse proxy. I've also used invalid URL's and it generates more/different errors.

is there any way you can log the incoming requests coming to the ollama? wanted to see which are the requests reaching ollama server and what was coming as payload..

we will try to replicate this setup and see what is the issue happening

[g7eg10r]

Logging the import.meta.env object shows that it is not populated as expected (OLLAMA_API_BASE_URL is an empty string, while there is a sensible value for this variable in .env file, Wrangler even sees it) :

when using the UI to set the baseUrl which I recommend it uses the browser cookies in both server and the frontend to get the configuration so these variables will be ignored

Indeed, disabling the cookie blocking extension on my web browser did let the configuration set at UI level propagate to the server side; thanks.

[Yet I face another kind of error with the following message in the container's log : "Uncaught (in response) Error: The script will never generate a response."].

[thecodacus]

can you provide a screenshot??

[k1rb]

When I load the page for the first time (with ollama url set via bolt UI), I get this in bolt container stdout:

[wrangler:inf] POST /api/chat 200 OK (174ms)
[WARN] Failed to get Ollama models:  Network connection lost.
[wrangler:inf] GET / 200 OK (205ms)
[wrangler:inf] GET /assets/ReactToastify-Bh76j7cs.css 200 OK (6ms)
[wrangler:inf] GET /assets/root-BKJfpClG.css 200 OK (10ms)
[wrangler:inf] GET /assets/tailwind-compat-Bwh-BmjE.css 200 OK (25ms)
[wrangler:inf] GET /assets/xterm-LZoznX6r.css 200 OK (25ms)
[wrangler:inf] GET /assets/index-BNYgI3gK.css 200 OK (27ms)
[wrangler:inf] GET /logo-dark-styled.png 200 OK (28ms)
[wrangler:inf] GET /assets/entry.client-FAOEGrxn.js 200 OK (30ms)
[wrangler:inf] GET /assets/root-IUcSKLvn.js 200 OK (29ms)
[wrangler:inf] GET /assets/index-DygJPlmn.css 200 OK (30ms)
[wrangler:inf] GET /assets/stripIndent-MRyAV8YL.js 200 OK (30ms)
[wrangler:inf] GET /assets/index-E26Bw2jv.js 200 OK (36ms)
[wrangler:inf] GET /logo-light-styled.png 200 OK (178ms)
[wrangler:inf] GET /assets/_index-DR27lJI9.js 200 OK (178ms)
[wrangler:inf] GET /assets/components-6Yp278Dk.js 200 OK (178ms)
[wrangler:inf] GET /favicon.svg 200 OK (3ms)
[wrangler:inf] GET /assets/wasm-CG6Dc4jp.js 200 OK (5ms)
[wrangler:inf] GET /assets/shellscript-HNM5thJl.js 200 OK (44ms)
[wrangler:inf] GET /assets/light-plus-w1el_4yc.js 200 OK (45ms)
[wrangler:inf] GET /assets/dark-plus-B74Y8nHz.js 200 OK (45ms)

In bolt with ollama selected no models are displayed in the dropdown. When I load the page I see no activity from ollama, I would expect to see a GET /api/tags maybe. If I enter text into bolt chat prompt with ollama selected and no model set I do see requests from ollama's side.

From ollama stdout:

[GIN] 2024/12/17 - 17:35:12 | 200 |    1.757872ms |      172.17.0.1 | GET      "/api/tags"
[GIN] 2024/12/17 - 17:35:12 | 404 |     680.206µs |      172.17.0.1 | POST     "/api/chat"

From bolt stdout:

✘ [ERROR] A hanging Promise was canceled. This happens when the worker runtime is waiting for a Promise from JavaScript to resolve, but has detected that the Promise cannot possibly ever resolve because all code and events related to the Promise's I/O context have already finished.


✘ [ERROR] workerd/server/server.c++:3613: error: Uncaught exception: workerd/io/io-context.c++:1207: failed: remote.jsg.Error: The script will never generate a response.

  stack: /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@35294c0 /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@352a9d0 /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@352b5d0 /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@2f03e90 /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@2f03f6e /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@359b35d /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@2ee4b00 /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@419688c /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5aa58af /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@4195280 /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@2ed9f10 /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@38d5e90 /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@36dacf0 /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@36db121 /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@5aa58af /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@352ac70 /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@352b5d0 /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@2f03e90 /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@2f03f6e /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@2f05fdc /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@41aa9e7 /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@41abbf2 /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@41ac69e /app/node_modules/.pnpm/@cloudflare+workerd-linux-64@1.20241106.1/node_modules/@cloudflare/workerd-linux-64/bin/workerd@4171f90


✘ [ERROR] Uncaught (in response) Error: The script will never generate a response.


[wrangler:inf] POST /api/chat 200 OK (261ms)

Hope this helps.

[g7eg10r]

can you provide a screenshot??

The container log :

app-prod-1  | [wrangler:inf] POST /api/chat 200 OK (667ms)
app-prod-1  | ✘ [ERROR] A hanging Promise was canceled. This happens when the worker runtime is waiting for a Promise from JavaScript to resolve, but has detected that the Promise cannot possibly ever resolve because all code and events related to the Promise's I/O context have already finished.
app-prod-1  | 
app-prod-1  | 
app-prod-1  | ✘ [ERROR] Uncaught (in response) Error: The script will never generate a response.

After setting up an appropriate snitch on Ollama's side, it appeared that Bolt was sending a request to Ollama for the "Claude Sonnet" model (although there was a single selectable Ollama model in the UI).
It may be the result of myself disabling all the providers but Ollama, for which no model was selectable (due to the previous issue), I just re-enabled Anthropic, re-selected my Ollama model and then it went fine (or so, now Ollama complains of missing Go of RAM to answer that looooong prompt!).

Thanks for your reactivity!

[g7eg10r]

@k1rb: it seems that your Ollama instance is responding "404" to the api/chat request (in the column after the date in your Ollama stdout dump).

May be is it because as no model was selected on the UI side, Bolt is sending a request w/ a model unknown to Ollama.

To troubleshoot, I used a socat-based container to dump requests on Ollama side, that would listen on port 11435 and relay traffic to Ollama on port 11434, and adapted the Ollama URL accordingly.