Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Wasm files that import JS resources #13608

Merged
merged 13 commits into from
Jan 1, 2023
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

### Fixes

- `[jest-runtime]` Support Wasm files that import JS resources [#13608](https://github.com/facebook/jest/pull/13608)

### Chore & Maintenance

### Performance
Expand Down
2 changes: 1 addition & 1 deletion e2e/__tests__/__snapshots__/nativeEsm.test.ts.snap
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Ran all test suites matching /native-esm-deep-cjs-reexport.test.js/i."

exports[`runs WebAssembly (Wasm) test with native ESM 1`] = `
"Test Suites: 1 passed, 1 total
Tests: 5 passed, 5 total
Tests: 6 passed, 6 total
Snapshots: 0 total
Time: <<REPLACED>>
Ran all test suites matching /native-esm-wasm.test.js/i."
Expand Down
11 changes: 11 additions & 0 deletions e2e/native-esm/__tests__/native-esm-wasm.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
*/

import {readFileSync} from 'node:fs';
import {jest} from '@jest/globals';
// file origin: https://github.com/mdn/webassembly-examples/blob/2f2163287f86fe29deb162335bccca7d5d95ca4f/understanding-text-format/add.wasm
// source code: https://github.com/mdn/webassembly-examples/blob/2f2163287f86fe29deb162335bccca7d5d95ca4f/understanding-text-format/add.was
import {add} from '../add.wasm';
Expand Down Expand Up @@ -54,3 +55,13 @@ test('imports from "data:application/wasm" URI with invalid encoding fail', asyn
import('data:application/wasm;charset=utf-8,oops'),
).rejects.toThrow('Invalid data URI encoding: charset=utf-8');
});

test('supports wasm files that import js resources (wasm-bindgen)', async () => {
globalThis.alert = () => {};
kachkaev marked this conversation as resolved.
Show resolved Hide resolved
jest.spyOn(globalThis, 'alert');

const {greet} = await import('../wasm-bindgen/index.js');
greet('World');

expect(globalThis.alert).toHaveBeenCalledWith('Hello, World!');
});
12 changes: 12 additions & 0 deletions e2e/native-esm/wasm-bindgen/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
/**
* Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/

// folder source: https://github.com/rustwasm/wasm-bindgen/tree/4f865308afbe8d2463968457711ad356bae63b71/examples/hello_world
// docs: https://rustwasm.github.io/docs/wasm-bindgen/examples/hello-world.html

import * as wasm from './index_bg.wasm';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused import?

Copy link
Contributor Author

@kachkaev kachkaev Nov 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks so, yes. All three files in this folder are auto-generated and I’ve decided to keep them as is. This will be helpful in the future if we need to re-generate the example again.

Maybe this like has its meaning actually. It makes sure that the wasm file is imported before JS, which may affect the result. index_bg.wasm and index_bg.js import stuff from each other, bg stands for bindgen.

export * from './index_bg.js';
141 changes: 141 additions & 0 deletions e2e/native-esm/wasm-bindgen/index_bg.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
/**
* Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/

import * as wasm from './index_bg.wasm';

const lTextDecoder =
typeof TextDecoder === 'undefined'
? (0, module.require)('util').TextDecoder
: TextDecoder;

const cachedTextDecoder = new lTextDecoder('utf-8', {
fatal: true,
ignoreBOM: true,
});

cachedTextDecoder.decode();

let cachedUint8Memory0 = new Uint8Array();

function getUint8Memory0() {
if (cachedUint8Memory0.byteLength === 0) {
cachedUint8Memory0 = new Uint8Array(wasm.memory.buffer);
}
return cachedUint8Memory0;
}

function getStringFromWasm0(ptr, len) {
return cachedTextDecoder.decode(getUint8Memory0().subarray(ptr, ptr + len));
}

function logError(f, args) {
try {
return f.apply(this, args);
} catch (e) {
const error = (function () {
try {
return e instanceof Error
? `${e.message}\n\nStack:\n${e.stack}`
: e.toString();
} catch (_) {
return '<failed to stringify thrown value>';
}
})();
console.error(
'wasm-bindgen: imported JS function that was not marked as `catch` threw an error:',
error,
);
throw e;
}
}

let WASM_VECTOR_LEN = 0;

const lTextEncoder =
typeof TextEncoder === 'undefined'
? (0, module.require)('util').TextEncoder
: TextEncoder;

const cachedTextEncoder = new lTextEncoder('utf-8');

const encodeString =
typeof cachedTextEncoder.encodeInto === 'function'
? function (arg, view) {
return cachedTextEncoder.encodeInto(arg, view);
}
: function (arg, view) {
const buf = cachedTextEncoder.encode(arg);
view.set(buf);
return {
read: arg.length,
written: buf.length,
};
};

function passStringToWasm0(arg, malloc, realloc) {
if (typeof arg !== 'string') throw new Error('expected a string argument');

if (realloc === undefined) {
const buf = cachedTextEncoder.encode(arg);
const ptr = malloc(buf.length);
getUint8Memory0()
.subarray(ptr, ptr + buf.length)
.set(buf);
WASM_VECTOR_LEN = buf.length;
return ptr;
}

let len = arg.length;
let ptr = malloc(len);

const mem = getUint8Memory0();

let offset = 0;

for (; offset < len; offset++) {
const code = arg.charCodeAt(offset);
if (code > 0x7f) break;
mem[ptr + offset] = code;
}

if (offset !== len) {
if (offset !== 0) {
arg = arg.slice(offset);
}
ptr = realloc(ptr, len, (len = offset + arg.length * 3));
const view = getUint8Memory0().subarray(ptr + offset, ptr + len);
const ret = encodeString(arg, view);
if (ret.read !== arg.length) throw new Error('failed to pass whole string');
offset += ret.written;
}

WASM_VECTOR_LEN = offset;
return ptr;
}
/**
* @param {string} name
*/
export function greet(name) {
const ptr0 = passStringToWasm0(
name,
wasm.__wbindgen_malloc,
wasm.__wbindgen_realloc,
);
const len0 = WASM_VECTOR_LEN;
wasm.greet(ptr0, len0);
}

export function __wbg_alert_9ea5a791b0d4c7a3() {
return logError((arg0, arg1) => {
// eslint-disable-next-line no-undef
alert(getStringFromWasm0(arg0, arg1));
}, arguments);
}

export function __wbindgen_throw(arg0, arg1) {
throw new Error(getStringFromWasm0(arg0, arg1));
}
Binary file added e2e/native-esm/wasm-bindgen/index_bg.wasm
Binary file not shown.
2 changes: 1 addition & 1 deletion packages/jest-runtime/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1684,7 +1684,7 @@ export default class Runtime {
const moduleLookup: Record<string, VMModule> = {};
for (const {module} of imports) {
if (moduleLookup[module] === undefined) {
moduleLookup[module] = await this.loadEsmModule(
moduleLookup[module] = await this.linkAndEvaluateModule(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's still added to the cache, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t know for sure unfortunately. There are calls to this._esmModuleLinkingMapget/set inside linkAndEvaluateModule but I don’t know what that means. All I know is that loadEsmModule cannot be used here – tests don’t pass.

Copy link
Member

@SimenB SimenB Nov 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to make sure the module is checked for in _esModuleRegister or what it's called. And then re-used or added afterwards

Copy link
Contributor Author

@kachkaev kachkaev Nov 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIU, the module already ends up in _esModuleRegister for wasm-bindgen packages. We have this import flow:

user-code.js
↓
[bindgen package] / index.js
↓
[bindgen package] / index_bg.js
↓↑
[bindgen package] / index_bg.wasm

Because index_bg.js is imported from index_bg.js, index_bg.js ends up in the cache registry by then. But because of circular deps, calling loadEsmModule instead of linkAndEvaluateModule fails because it returns an object with status: 'linking' instead of a string path.

It’s possible to imagine a Wasm file that imports from some other JS file. I don’t know what will happen in this case TBH. Perhaps we can focus on fixing a practical problem in this PR and leave this theoretical scenario for later. Wasm support is still experimental anyway.

Feel free to push changes if you can think of any! The key added value of this PR so far is a new test case, not the fix.

await this.resolveModule(module, identifier, context),
);
}
Expand Down