Example of Default Output that is compatible for Node and Browsers

[mbalex99]

In the main README.md, it states:

By default the generated JS uses ES modules and is compatible with both Node and browsers (but will likely require a bundler for both use cases).

But it seems like the ES module can't be consumed by node directly. Like I can't do this:

const myLib = require('myLib')
const sum = myLib.add(1, 2)
console.log(sum)

However the --nodejs flag for the compiler allows this to work seamlessly. Do we need to use webpack for both browser setup and node setup with the default es6 modules?

Ideally, it would be fine if the default usage in the browser would need webpack, but what is the ideal build setup for using it in nodejs without webpack?

[fitzgen]

Would love to receive a PR that clarified the README!

Do we need to use webpack for both browser setup and node setup with the default es6 modules?

I don't know how node plans on supporting es6 modules, but maybe @ashleygwilliams knows more.

Ideally, it would be fine if the default usage in the browser would need webpack, but what is the ideal build setup for using it in nodejs without webpack?

If you aren't supporting the Web, and are only supporing node, then the --nodejs flag should work fine for you. There is also the --no-modules flag to do a UMD-style thing.

[mbalex99]

Well node supports es6 modules if you do this:

node --experimental-modules myFile.mjs 

However the problem is in the top of the generated bindgen file you have something like:

import * as wasm from './mylib_bg';

Here are the files that it generated (default settings):

• mylib.js
• mylib.d.ts
• mylib_bg.wasm

and running:

$ mv mylib.js mylib.mjs 
$ node --experimental-modules myFile.mjs 

Results in:

(node:98839) ExperimentalWarning: The ESM module loader is experimental.
{ Error: Cannot find module ./mylib_bg
    at search (internal/modules/esm/default_resolve.js:28:12)
    at Loader.resolve [as _resolve] (internal/modules/esm/default_resolve.js:64:11)
    at Loader.resolve (internal/modules/esm/loader.js:56:18)
    at Loader.getModuleJob (internal/modules/esm/loader.js:88:40)
    at ModuleWrap.promises.module.link (internal/modules/esm/module_job.js:35:40)
    at link (internal/modules/esm/module_job.js:34:36) code: 'MODULE_NOT_FOUND' 

It'd be great if the default output of wasm_bindgen accomodated for both commonjs and es6 module with webpack. Since commonjs isn't going away anytime soon. :-\

[FreeMasen]

I spent a good amount of time on this problem ~ a week ago. Looking over what google has to say about writing js for both Node and the browser is pretty tricky.

The biggest issue right now is that node isn't currently resolving .wasm files with require instead we need to use require('fs') to read in the Buffer and then compile it to a wasm module. To further complicate this simply including require('fs') anywhere in your script will cause webpack to report an error since it can't resolve fs.

There isn't a good test that I can find to account for this

if (require) {
  //node
} else {
  //browser
}

The above doesn't work since webpack would attempt to bundle any of the code you included in either side of the block, which would again attempt to resolve fs.

I think there might be a way to get this to work using eval, something like

const nodeWasm = `...`;
module.exports = function() {
    if (typeof self !== 'object') {
        return eval(nodeWasm);
    }
    return require('./module_bg.wasm');
}()

but I haven't been able to test that as of yet

[mbalex99]

You hit all the painpoints we did. Eventually I had to create a post build script to remove node related files. It feels super janky but it was the only way that webpack wouldn't try to read files we didn't want it to. :-/

[FreeMasen]

After some additional work I was able to get the eval solution to actually work for both environments. It is actually pretty weird to me that webpack can resolve require('util') in the main bindgen file but here we are.

The full solution can be found here;

In the pkg folder you can see that I have taken the TextEncoder from the default file and the exports from the nodejs file. Then in the _bg.js file I have defined a resolve function that executes the file system actions inside of an eval, only when typeof self !=== 'object'. It may not be the prettiest solution but it does work.

At this point I think that instead of updating the default bindings to use the method that a new flag get added to direct the cli tool to use this method. Using eval feels kinda hacky so I am not sure we want to go that route. Ideally I would like to see the target flags be --nodejs, --dual-env (or similar) with the default the browser version since webpack is doing the heavy lifting for us there.

I might be able to dig in on this next week.

[xtuc]

If we use the experimental esm support in Node, it would be interesting to demonstrate https://github.com/wasm-tool/node-loader too?

[alexcrichton]

The intention of wasm-bindgen is that the contents of the JS file, by default, are compatible with both Node and browsers. The main caveat is that neither (unfortunately) supports ESM today by default (although browsers may soon). In that sense I think this is primarily related to how JS imports are defined, right? If so @xtuc's suggestion may be a great way to go!

[xtuc]

That make sense @alexcrichton.

I'm part of the wg for ESM in Node and indirectly for browser as well, interop between CJS and ESM is a concern there. I think that using ESM by default is the way to go since they are usable from CJS and will be default in the JS ecosystem at some point.

@mbalex99 Webpack supports now ESM by default, I don't see an issue there.

---

One issue is see is for Node without the ESM support, in that case it's necessary to transpile ESM down to CJS (using Babel?). I would be happy to help with that.

[brendankenny]

It seems like there are two issues going on:

• --no-modules could support both the browser and node if the individual --browser and --nodejs flags aren't given. Today --no-modules supports just browsers, but with a simple environment conditional it could require() instead of using global utilities, fs.readFileSync() instead of fetch(), and write to module.exports instead of self.wasm_bindgen if in node.

  That would support everywhere today, and using the file in a bundler like webpack is still well supported without eval trickery. If it's going to run in node, use target: node and it'll use the native core modules. If it's for the web, use target: web or webworker and the default stubs will keep the core modules like fs out of the bundle and the runtime checks mean they won't be touched.

• the other issue seems to be import * as wasm from './mylib_bg';. It is nice that webpack supports that syntax, but considering that expanded load functions are already maintained for the browser and node anyways (used in --no-modules and --nodejs modes, respectively), and the fact that importing wasm as a module is still non-standard, it seems like that could be dropped for now.

  That would allow use in basically all browsers with wasm support today and in node if run with --experimental-modules. And again, webpack support is trivial if target is set correctly.

Doing both of these would hopefully simplify a few things and allow the library to be more independent of any specific bundler while still being fully compatible with them. It should also make documentation/getting started simpler as webpack wouldn't be required to get started but would work fine if/when the user decides to use it in their pipeline.

[xtuc]

@brendankenny thanks for taking the time to clarify that and I agree. Similar discussions are going on in a few other places, let's try to keep them in one place.

It seems that a Webpack loader for wasm-bindgen is the way to go 😇

It might be worth going through the RFC process https://github.com/rustwasm/rfcs to have a clear idea of what to do here.

and the fact that importing wasm as a module is still non-standard, it seems like that could be dropped for now

While I agree, I don't think we should drop it. That's the goal we're aiming with https://github.com/WebAssembly/esm-integration/.

---

One thing to keep in mind is that we don't want to add a JS dependency in wasm-pack. I assumed that using Babel or something would be ok but a flag is the way to go.