[FEATURE] Integrate with WASM for HTML5 Web Builds

[DomPTech]

Is your feature request related to a problem? Please describe.
I'm currently making a game using this addon for a competition, which requires that I submit a web build.

Describe the solution you'd like
There is already an existing solution for integrating LlamaCPP with WASM, found here: https://github.com/tangledgroup/llama-cpp-wasm. I'm not familiar enough with how plugins work, but the README suggests that implementation into this addon should be fairly simple.

Describe alternatives you've considered
N/A

Use case
This is necessary to support HTML5 web builds, which is arguably the most accessible format for both users and beginner developer uploading to Itch.io.

Additional context
N/A

[AsbjornOlling]

I think this is a good idea.

There is also wllama, another project to provide wasm bindings for llama.cpp

but the README suggests that implementation into this addon should be fairly simple.

I don't agree. I don't think that this will be a quick and easy thing to do.

If you know something that I don't I'll be glad to see a PR that implements this stuff in a fairly simple way 😇

HTML5 web builds, which is arguably the most accessible format for both users and beginner developer

Completely agree. Web builds are great!

There are at least couple of tricky things:

• do we provide a parallel implementation of everything for wasm (difficult to maintain), or do we add wasm support to llama-cpp-rs. How to do wasm FFI in rust? Do the types match the types of C++ FFI?
• how to deal with model splitting. IIRC, models larger than 4GB need to be split into multiple files. Do we provide a list of paths to the ModelNode? A pattern to match?
• exports with model files are already weird on desktop platforms. how weird will they be on web? What does godot's globalize_path do on a resource path when running in the web browser?
• what to use for acceleration? CPU is slow, but Webgpu isn't widely supported yet
• are people even okay with downloading a webpage thats several gigabytes in size?

[DomPTech]

I absolutely agree with all of your points. After testing an existing demo, I've realized it can take upwards of 20 seconds to load even very small models like TinyLlama-1b. Simply put, this would be terrible for UX. While costing real money, it's probably easier (and faster) to make an API call to OpenAI or Google Gemini. Client-side resource constraints already make <100MB web applications slow to load, especially in a 3D environment with physics and multi-threading. A server-side solution overall offers better performance, scalability, and interfacing with additional capabilities like function calling. At the moment, while it's an amazing free alternative, I just don't see LlamaCPP being feasible on the user side.

[AsbjornOlling]

I'm going to cheekily re-open this, because I think it is something that would be meaningful and valuable to add sometime in the future.

It's also useful to have this issue as a place to refer to for notes and research :)