Performance problem when I copy cells?

[Gomez12]

I could try to fix it by myself and add it as a PR, but I can't seem to find the problem.

But I use a litellm llm proxy to control my requests. Which I have added as an OpenAICompatible llm.
And this utility works perfect for small amounts of cells.

But it looks to be doing something in excel which is extremely slow.

If I add a prompt(a2,"Tell me the value given") then it works perfectly on one cell.
But if I copy that over 25.000 cells then it just stays on #N/B status.
My litellm proxy does not receive any requests as well. (So it is not my llm slowness)

If I copy it to a 100 cells then it stays a minute on #N/B and then it starts filling the cells.
To me it looks like it is something on the excel side, but I can't seem to find it (as I have no experience profiling / debugging excel addins).

Is there any info you could give where I could look for it?

[kaspermarstal]

Hi @Gomez12, yes this is a problem we must fix, thanks for pointing it out.

The problem is that Excel's main thread is blocked when we read out cell values to build the prompt that will be sent to your llm. We try to read them out as quickly as possible so we can release the main thread quickly. The most recent PR #169 is part of the effort to improve how quickly we can unblock the main thread.

If you are up for it, you can compile the main branch yourself and get the most recent speedups. If not, I'm afraid you must wait until the next release to benefit. The problem will still remain though, especially for 25.000 cells! Sounds cool, what are you working on?

[kaspermarstal]

Hi, after a bit of digging we managed to get the overhead per request down to ~1 ms per request. Still, with 25.000 requests, it will take ~25s on the main thread to queue the requests. We did some work to mitigate thread pool starvation which we think was the main cause of the unresponsive UI. I made a build for you and an installer. I hope you are up for testing it.

To try it out:

1. Uninstall your current version of Cellm (see https://docs.getcellm.com/get-started/install#uninstall)
2. Close and re-open Excel and confirm that Cellm is indeed uninstalled.
3. Download the file below, extract, and run the installer.
4. Open Excel and run your prompt on 25.000 cells. You will have to setup your model endpoint again.
5. Let me know how it goes! If can share a gif/video, it will be easier to help you further.

Cellm-AddIn-Release-x64.zip

[kaspermarstal]

FYI @Gomez12 we made significant performance improvements since you opened your issue and even since I made the installer for you. Closing the issue now, but feel free to open if a build from the main branch is still slow.