WASI Libraries

[CryZe]

So I'm using wasm with wasmer + cranelift for a kind of scripting system for my application. So users can write little WebAssembly modules to write plugins. Over the weekend I've looked into integrating WASI to give them access to the operating system in a limited fashion (I'm planning to only let them read files and print to stdout / stderr initially). However it seems like so far (at least from Rust, not sure if this is an overall limitation) you can really only create WASI binaries, not libraries. The way my plugin system works is that my application expects certain exports to be there and I call into them, while providing WASI imports + other plugin specific imports to the wasm files. However at least the way it currently is in Rust and the wasi libc sysroot, WASI really only works for the duration of a main function that needs to be there. Calling the wasi functions from anywhere else (through the Rust std) segfaults because libpreopen isn't initialized (anymore). I'm not sure how much this is a design limitation or a just a bug, but overall I'd like my main application to have control over which functions to call. With a main function, the WebAssembly module takes control over the control flow, which isn't great for such an "extension" kind of use case.

[ericsink]

I bumped into this problem as well, although from C instead of Rust, so I'm not sure how of much of this applies to your situation.

Yes, it appears that _startup() initializes libpreopen before it calls main(), so if I don't want _startup() or main(), I have a problem.

I'm assuming that the answer is that if I compile with

-Wl,--no-entry

as an argument to clang, then I am supposed to take responsibility for initializing libpreopen myself.

There seem to be functions in <wasi/libc.h> and <wasi/libc-find-relpath.h> to be used for that purpose.

But I haven't actually tried using them yet.

[PoignardAzur]

Wait, is preopen necessary for calling WASI "syscall" functions? I thought it was only used as a polyfill for standard library calls using absolute paths?

If you want to make a plugin system (eg, for a text editor) that passes directory capabilities to its plugin and doesn't allow absolute paths, do you still need preopen? Or the standard library, for that matter?

[CryZe]

The main reason I'm even using WASI is to be able to open and read from files in the plugins. Not using the standard library for that would probably also mean that I just don't need WASI. But I feel like that defeats the purpose of WASI if you can't even use it in such use cases.

I feel like the standard library should either lazily initialize preopen if you don't go through main, or there should be __wasi_init() and __wasi_deinit() exports in case you use it as a library, which the runtime can use instead of main to initialize preopen and whatever else the WASI module needs.

[sbc100]

I agree, it does seem reasonable to be able to initialize wasi without going through crt1.o.

[kitlith]

Well, with elf binaries, the main difference between an application and a library is the presence of _start. (Sometimes, even, a library is also used as an application)

With elf libraries, if any init needs to be done before you can call library functions, it's put in the .init and .fini sections (or .init_array and .fini_array). If we did something similar, it could help solve the issues with libpreopen even if it wouldn't really help with malicious plugins.

Discussion in #19 may also be relevant to this issue,

[kitlith]

As a workaround for the issues with libpreopen, it may be possible to get the fd that has been assigned to a directory (if that functionality has been exposed in the support code! wasmer-wasi does not currently), pass it to a plugin through , use path_open (aka openat, there's a rust crate for this for example), and then proceed using the standard library once a regular file has been opened.

This is not the greatest, but it may be workable.

[newpavlov]

Note that this issue could be used by a pure-Rust implementation of open_parent, e.g. as implemented in rust-lang/rust#64434, but unfortunately that PR got closed.

[jpmartin2]

If I'm not crazy, I think WebAssembly/wasi-libc#74 might help solve this problem, but I have absolutely no idea how wasi-libc gets pulled in or how to force rust to use a newer version.

[jpmartin2]

(copy pasting what I posted rust-lang/rust#73432 (comment))

wasi-libc actually already has a change to help with this (WebAssembly/wasi-libc#74), so at this point I think it's just a matter of rust taking advantage of it. And I managed to figure out how to hack the rust compiler to get it to work, involved setting wasi-root in rust's config.toml (to point to a recent build of wasi-libc) and changing the crt object linked against for the wasi target in librustc_target to crt1-reactor.o instead of crt1.o. Then I was able to produce a wasm file using a binary target with #![no_main] and rustflags = ["-C","link-args=--no-entry"] (creating a cdylib didn't work), and got a wasm file with an _initialize export which does the libpreopen setup (this is provided by crt1-reactor.o). I was able to run the wasm file using wasmtime and call other functions that did file io successfully (I was able to get it working in wasmer as well, though wasmer doesn't call _initialize for you like wasmtime does right now, so you have to add that call yourself).

For clang, it looks like there's already a patch https://reviews.llvm.org/D62922 that allows for something similar, though I don't know if that's in any releases yet and I haven't tried it at all myself.