Add WASM (WASI) host support (including, but not limited to, the HTML5 target)

[Type1J]

Describe the project you are working on:
I am working on an app that needs updates often due to new features.

Describe the problem or limitation you are having in your project:
Replacing a GDNative dll is how we currently need to distribute new updates. Each platform needs it's own dll. The dlls are written in Rust. GDScript isn't an option due to the CPU intensive tasks in other threads that we are performing.

Describe how this feature / enhancement will help you overcome this problem or limitation:
Only 1 .dll (.wasm) file will need to be produced, and it will work on all platforms

Show a mock up screenshots/video or a flow diagram explaining how your proposal will work:

Current:
Windows -> .dll
Linux -> .so
macOS -> .dylib
HTML5 -> Currently not supported, but desired
Android -> .so
iOS -> Currently not supported, but desired (for our project)

After proposal:
Windows, Linux, macOS, HTML5, Android, and iOS (potentially more) -> .wasm

Describe implementation detail for your proposal (in code), if possible:
The .wasm dynamic library would basically implement the same API as GDNative dynamic libraries currently do. See wasmer and WASI (https://wasi.dev/) for an example host native application.

If this enhancement will not be used often, can it be worked around with a few lines of script?:
No. Currently, this is a build system and distribution problem. The feature is the only option for a GDNative library for HTML5, and it can be used to prevent the need for other proposals to go into the core.

Is there a reason why this should be core and not an add-on in the asset library?:
It enables asset libraries to make "native"-ish extensions that can be used on multiple platforms. They may not be as fast as true native, but they are faster than GDScript by far, and enable legacy code in the same way that GDNative does, but WASI also provides sandboxing support that GDNative does not have, which reduces the security audit required in some cases with asset library GDNative dynamic libraries.

Bugsquad edit (keywords for easier searching): WebAssembly

[Calinou]

See also godotengine/godot#28303.

[Frontrider]

You could try using this:
https://wasmtime.dev/
This is a standalone webassembly vm, with a lot of embedding options including C, WASI included.

This one also means that Godot would supports a huge number of languages without any specific runtimes. Debugging is a real question tough, as I don't know if webassembly has any way to map the bytecode back to whatever it was compiled from.

[robbiespeed]

As a stepping stone why not keep the existing structure of compiled per platform libraries, but add wasm to the list for the web platform. Later it could be tested weather allowing wasm to be used for all platform builds via some wasm runtime should be added as an option, by testing the performance impact that would introduce.

[Type1J]

We are already using a Rust GDNative plugin that includes wasmer-runtime and loads a .wasm library that is switched out. We still don't support HTML5, but we've worked around this issue for now. I believe that it should be resolved before we expand to HTML5 and iOS.

[Frontrider]

@robbiespeed Things that are designed for the web are always fast. That's one of the main business targets. Faster execution means that your customer gets to your products sooner. Overly optimized javascript is not that much slower than native code (the parsing is the part where it has a small slowdown, yes I was surprised), and webassembly is even faster than that.
https://takahirox.github.io/WebAssembly-benchmark/

[robbiespeed]

@Frontrider yes wasm is faster than javascript (in some workloads), and we have come a long way in performance on the web, but it is still slower than native compiled code.

Here's a fairly in depth paper on wasm vs native performance https://arxiv.org/pdf/1901.09056.pdf

In that paper it describes that on average in their tests wasm performed roughly 1.5x slower than native.

[Frontrider]

@robbiespeed I know that it's slower. Nothing is faster than native at this point. I personally don't think speed is going to be an issue for now (it's already faster than whatever runtime this small-ish team could produce on their own), and the plan is to compile gdscript to webassembly and that would be a big speedup on it's own. (I'm personally fond of gdscript, even tough I do not like python's syntax)
The benefits of not needing to write any specific compatibility layers for a large chunk of languages/tools is also big one.

And yes I agree, GDnative should remain and be improved, because there is a clear usecase for it.

[Calinou]

and the plan is to compile gdscript to webassembly and that would be a big speedup on it's own.

I don't think that's currently planned.

[robbiespeed]

@Frontrider this proposal if I interpreted it correctly is about introducing wasm as a build target for gdnative projects, and replacing existing gdnative generated platform specific libraries with wasm, so that you only need to have one build target.

I am very much looking forward to the first part, because we need that in order to have web as a supported build target for gdnative projects. I don't think the second part is a good idea though unless it were optional, even then I don't see much benifit.

[TheRawMeatball]

This seems like the place where HTML5 support for gdnative is tracked, is this feature (primarily the first part) on the roadmap?

[Calinou]

@TheRawMeatball Support for GDNative in HTML5 may happen in the future, but there's no target version.