webassembly branch: generating wasm, input buffer

[rajeevgUCI]

I successfully used the webassembly branch to compile a pipeline to wasm, and ran the pipeline on data in the browser. But I was wondering if my approach can be improved: is there a better way to convert a Halide pipeline to wasm, or a better way to generate an input buffer?

My current approach is:

1. Compile Halide pipeline to .s assembly using the webassembly branch.
2. Use binaryen to convert the .s to .wasm. I had to make a change to the webassembly branch, see my commit here.
3. Write C++ code that creates a Halide input buffer.
4. Use emscripten to compile the C++ code to .wasm.
5. Write JavaScript glue code between the Halide-generated wasm and the emscripten-generated wasm.

Thanks!

[steven-johnson]

The WASM branch is ~1 year old so I'm not surprised that it could use a little polish around the edges. (IIUC there hasn't been a lot of demand expressed for it hence the lack of work on it.) We'd definitely welcome additional work on the branch to make it easier to fit into things.

[zvookin]

I mostly decided to stop working on WebAssembly until things rationalized somewhat due to issues like this. It was not obvious how to utilize the technology and I don't really have a good use case so I can't really experiment to figure it out even if I had the time.

Glancing over the linked issue and some things hanging off it, I'm unconvinced much progress has been made in terms of what they're doing. The whole thing strikes me as pretty dubious to be honest.

That said, it seems like we want to surface both the ELF and s2wasm options. One way to do this is to have compile_to_object produce the ELF variant and compile_to_assembly produce the s2wasm input variant. (EDIT: I probably have this mapping backward. It seems "-wasm" actually produces an object file LLD might be able to link where ELF changes the assembly syntax to something s2wasm```` can assemble. (Contrary to intuition, wasm in many contexts, e.g. s2wasm```, is not assembly (.s) but object code (.o).)

I'm not sure the above way of surfacing the difference is a good idea, but it might be. It requires adding some more plumbing to the Halide compilation path which is a bit ugly, but not too hard to do.

What is your use case? Are you seeing useful results from Halide?

[zvookin]

Oh, the other thing would likely be to add automatic generation of JavaScript glue to call the WebAssembly. The API design for this is not really my area of expertise. It would sort of be an equivalent to compile_to_header for C++ I think.

[zvookin]

For now, I think I'm going to throw in a total hack and overload "NoOS" as the operating system to mean using the wasm target triple and default to ELF otherwise per the commit.

[rajeevgUCI]

My "big picture" use case is to see if I can use Halide with WebAssembly to improve neural network performance in web browsers. This is part of a research project at UC Irvine with @sajjadt.
The first, basic step is to run a Halide pipeline on an image in the browser. I was able to successfully do this, but am trying to improve the process.

s2wasm and ELF problem:
I recently found out that s2wasm has been removed from binaryen; see this GitHub issue. It seems that LLVM tools are recommended instead. I was able to use the LLVM tools to compile the pipeline to a wasm module, without needing to make the ELF changes. So I sidestepped the s2wasm and ELF problem, and am hoping that the LLVM tooling is less subject to change.
My build steps now look like this:

g++ halide_func_generator.cpp -I $(HALIDE_LIB)/include/ -L $(HALIDE_LIB)/bin/ -lHalide -lpthread -ldl -o halide_func_generator -std=c++11
./halide_func_generator # outputs halide_func.bc
$(LLVM_LIB)/build/bin/llc -march=wasm32 -filetype=obj halide_func.bc -o halide_func.o
$(LLVM_LIB)/build/bin/wasm-ld halide_func.o --allow-undefined --no-entry --import-memory -o halide_func.wasm

Is this a more stable process that works better with Halide?

Automatically generating JavaScript glue code:
The wasm module generated by the above imports some Halide error functions (e.g. halide_error_access_out_of_bounds) and takes halide_buffer_t arguments.
Implementing the error functions in JavaScript and having the wasm module import them is straightforward.
The problem is how the JavaScript can provide the halide_buffer_t arguments. My current solution is:

1. In JavaScript, create JavaScript TypedArrays for input and output.
2. Emscripten-compiled C++ code wraps a TypedArray in a halide_buffer_t.
3. That halide_buffer_t is returned to JavaScript. The JavaScript passes it to the Halide wasm.

But, I have not yet figured out a way to generate all of that automatically.

[zvookin]

@steven-johnson @rajeevgUCI IS this issue still relevant given the merged WebAssembly support? It looks like there may still be a usability issue in integrating Halide wasm output with the JavaScript. If so, it seems it may be more productive to open a new issue with that made explicit and to close this one.

[steven-johnson]

Sorry, I never saw the comment from @rajeevgUCI.

(1) Halide only supports the llvm backend; we don't support fastcomp, s2wasm, etc and likely never will, as the LLVM backend is where most dev work is happening now.

(2) we haven't investigated autogenerating JS glue code, as (literally) no one else has requested it so far, believe it or not. (we'd be very happy to have a PR contributing such a change.)

agreed that closing this issue and opening new one(s) as appropriate is the right thing at this point.