Use case: Web support using Emscripten

[kripken]

(Background: This reddit post where @andrewrk suggested I file this issue.)

I think it would be useful for Zig to allow compiling the same code both natively and to the Web.

Zig already supports the Web to some extent with the wasi and freestanding wasm targets. The difference with Emscripten is in API support: Emscripten allows common native APIs to be compiled to the Web, such as SDL2, OpenGL, OpenAL, pthreads, and so forth. A very common use case for Emscripten is to port a game engine, and that's why Unity and many others use it. As game engines begin to be written in Zig, allowing the same there as for C++ would be nice I think. (Of course, this isn't limited to games: Photoshop, Figma, AutoCAD, etc. also use Emscripten.)

What this means concretely: this gist has an example of a C program that uses GLES3 and GLFW to render, and relies on Asyncify to handle the Web event loop. That exact same program can be compiled with gcc or clang natively, or with emcc to the Web - that's the big benefit here.

The gist also has an example of a Zig program that I managed to do the same with, with some hacks. With small (I hope) improvements to Zig I think the hacks could be avoided, namely to allow setting the Emscripten target triple, and to get the full Zig stdlib compiling with that triple (hopefully simple, as the differences between the Emscripten wasm triple and the WASI wasm triple are very small). Emscripten itself would do all the rest, when the user runs it on the object files emitted from Zig (see details in that gist; there may also be more interesting Zig build system integration opportunities, but I don't know enough about that).

Why is it useful to compile the exact same code both natively and to the Web? You don't have to, of course - you can have separate codebases for the two platforms. For example, you could add a new rendering backend to your game engine to use WebGL, you can rewrite all your event handling logic to be async like the Web requires, rethink how you handle files, etc., and all the other stuff that is different on the Web. Such rewrites may have benefits, sometimes. But it makes porting harder, it makes debugging harder, and you may end up handling the mismatches between native and the Web in a sub-optimal way. Reusing the large effort put into Emscripten here can be useful.

[kubkon]

I'm happy to look into this as I've got a running example of a cross between C++ that requires both threading and exceptions, and pure Zig code. Ideal for testing the emscripten target triple.

[KioriSun]

Hey @kripken thanks for taking the time to help us!
Tried the gist and didn't work, both c and zig examples.
They run, but the screen stays black.

Interesting non-essential observations:
The initial obj gen by zig is around 5 times larger than the clang one, yet the final wasm is only around 5 kb larger.
The js and html emitted are seemingly the same size.
When editing the zig version ZLS(I think) complains about the "//" at the beginning of the shader string.

I will try again later on with my own code, just in case.
I was already going to test gles2 anyways, for webgl 1 support, which I think is more widely available.

Let me know if there is anything you'd like me to test.

I have latest emcc and zig.

[kripken]

@KioriSun Oh, sorry, looks like I forgot to write that you need to manually add _run_zig() in the HTML, or do that in the dev console. That is, the zig code can't use the automatic startup code because that isn't integrated between zig and emcc. Does that fix things for you? If not, is there an error in the dev console?

[KioriSun]

_run_zig()

Thanks it works. I was wondering about the -s STANDALONE_WASM -s EXPORTED_FUNCTIONS=_run_zig --no-entry part of the code and why it was necessary, how it would be used.
I'm not an expert on zig internals, but hopefully that integration can be reached.

A hint for everyone else that might give this a shot:
Another thing that I did that is important is i didn't just run the files but served them from a file server.
(this prevents cors issues and all, that modern wasm requires)

I couldn't get the _run_zig() automated, tried adding it to all sorts of onload functions, didn't work. only running from the console so far. Now that it's at least running form the console, I wanna try other apis. But I assume they'll run fine.

About automating the run function, for the time being, couldn't a properly placed EM_JS call solve that?
I'm not sure when/where the run function exists, as stated I tried calling it in other contexts and it either didn't exist or was undefined.

[kripken]

hopefully that integration can be reached.

Yes, exactly - that's one of the things that I think could be worked on, as part of this topic.

About automating the run function, for the time being, couldn't a properly placed EM_JS call solve that?

Hmm, no, because EM_JS happens after startup.

But a generic solution to this could be to add this to the HTML:

Module.onRuntimeInitialized = function() {
  _run_zig();
};

That uses one of the emscripten JS startup hooks. But this is kind of a hack that we should fix with proper integration in the Zig and emcc startup code, as mentioned before.

[KioriSun]

Module.onRuntimeInitialized = function() {
  _run_zig();
};

This is exactly what i was looking for in terms of lifetimes. But it didn't work for me. Maybe I'm missing something. I tried adding it into a script tag, body onload, and adding it directly into the zig.js that emcc builds. Nothing seems to work only running from the console.

If EM_JS happens after "startup", it also happens before or after "onRuntimeInitialized"? Because this is what I thought might work as a in code hack because if the objs are already there, maybe the function can be added, maybe it can't.

[floooh]

I can 'contribute' a little real-world example here for testing:

https://github.com/floooh/pacman.zig

WASM version here:

https://floooh.github.io/pacman.zig/pacman.html

Build instructions:

https://github.com/floooh/pacman.zig#experimental-web-support

The project is a mix of cross-platform C headers which makes use of Emscripten APIs and EM_JS() feature, and platform-agnostic Zig code for the gameplay stuff.

To make the build work I had to compile the Zig parts with wasm32-freestanding instead of wasm32-emscripten, otherwise I would get build errors in Zig standard library. Here's the "hack":

https://github.com/floooh/pacman.zig/blob/549a73ecd6f5c9bfe4f8150b08e4b43f02eae331/build.zig#L88-L91

To reproduce the problem just comment out this line:

https://github.com/floooh/pacman.zig/blob/549a73ecd6f5c9bfe4f8150b08e4b43f02eae331/build.zig#L89

...trying to build then produces this error:

/opt/homebrew/Cellar/zig/0.9.0/lib/zig/std/start.zig:318:17: error: unsupported arch
        else => @compileError("unsupported arch"),
                ^
/opt/homebrew/Cellar/zig/0.9.0/lib/zig/std/os.zig:134:24: error: container 'std.os.system' has no member called 'fd_t'
pub const fd_t = system.fd_t;
                       ^
/opt/homebrew/Cellar/zig/0.9.0/lib/zig/std/fs/file.zig:31:26: note: referenced here
    pub const Handle = os.fd_t;
                         ^
/opt/homebrew/Cellar/zig/0.9.0/lib/zig/std/fs/file.zig:15:13: note: referenced here
    handle: Handle,
            ^
/opt/homebrew/Cellar/zig/0.9.0/lib/zig/std/debug.zig:290:34: note: referenced here
                const stderr = io.getStdErr().writer();

Cheers!

[floooh]

PS: as I learned from this thread (https://groups.google.com/g/emscripten-discuss/c/fWPEAulgslM), building the C code with wasm32-emscripten is required to make the "EM_JS() magic" work. The Javascript source string is placed into a global variable which gets some special treatment by Clang only if the wasm32-emscripten triple is set. The string is then extracted from the object files by the Emscripten linker and placed into the Javascript file that's produced along with the .wasm and .html output.

I think the biggest motivation to better support wasm32-emscripten in Zig is that this could pave the road for an EM_JS()-like feature directly in Zig :)

...ideally at some later point it would be great if such special Emscripten linker features could be somehow integrated into the Zig toolchain, or if this is 'out of scope" maybe a better way to integrate a slim EMSDK (just the linker, sysroot and JS shims) with a Zig build.

[kripken]

@KioriSun

But it didn't work for me. Maybe I'm missing something. I tried adding it into a script tag, body onload, and adding it directly into the zig.js that emcc builds. Nothing seems to work only running from the console.

It's possible that adding that hook in the wrong place will not work. Here is what works for me locally, this is near the end of the HTML file:

        };
      };
      Module.onRuntimeInitialized = function() { _run_zig() }; // this is the new line
    </script>
    <script async type="text/javascript" src="a.js"></script>

[KioriSun]

Module.onRuntimeInitialized = function() { _run_zig() };

Thanks a lot! I found the issue. All of the above works if you do ; Module.onRuntimeInitialized = function() { _run_zig() };.
Mind the ;. It seems that at the end of the js script in the html, emscripten doesn't emit a last, trailing, ";", and that's why it wasn't working before.