Skip to content

guide: Add documentation for testing with wasm-bindgen-test #818

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 12, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
164 changes: 2 additions & 162 deletions crates/test/README.md
Original file line number Diff line number Diff line change
@@ -1,167 +1,7 @@
# `wasm-bindgen-test`

This crate is an experimental test harness for `wasm32-unknown-unknown`, with
the goal of allowing you to write tests as you normally do in Rust and then
simply:

```
cargo test --target wasm32-unknown-unknown
```

This project is still in the early stages of its development so there's not a
ton of documentation just yet, but a taste of how it works is:

* First, install the test runner.

```
cargo install --path crates/cli
```

(this comes with the normal `wasm-bindgen` CLI tool

* Next, add this to your `.cargo/config`:

```toml
[target.wasm32-unknown-unknown]
runner = 'wasm-bindgen-test-runner'
```

* Next, configure your project's dev-dependencies:

```toml
[dev-dependencies]
# or [target.'cfg(target_arch = "wasm32")'.dev-dependencies]
wasm-bindgen-test = { git = 'https://github.com/rustwasm/wasm-bindgen' }
```

* Next, write some tests!

```rust
// in tests/wasm.rs
extern crate wasm_bindgen_test;

use wasm_bindgen_test::*;

#[wasm_bindgen_test]
fn pass() {
assert_eq!(1, 1);
}

#[wasm_bindgen_test]
fn fail() {
assert_eq!(1, 2);
}
```

* And finally, execute your tests:

```
$ cargo test --target wasm32-unknown-unknown
Finished dev [unoptimized + debuginfo] target(s) in 0.11s
Running /home/.../target/wasm32-unknown-unknown/debug/deps/wasm-4a309ffe6ad80503.wasm
running 2 tests

test wasm::pass ... ok
test wasm::fail ... FAILED

failures:

---- wasm::fail output ----
error output:
panicked at 'assertion failed: `(left == right)`
left: `1`,
right: `2`', crates/test/tests/wasm.rs:14:5

JS exception that was thrown:
RuntimeError: unreachable
at __rust_start_panic (wasm-function[1362]:33)
at rust_panic (wasm-function[1357]:30)
at std::panicking::rust_panic_with_hook::h56e5e464b0e7fc22 (wasm-function[1352]:444)
at std::panicking::continue_panic_fmt::had70ba48785b9a8f (wasm-function[1350]:122)
at std::panicking::begin_panic_fmt::h991e7d1ca9bf9c0c (wasm-function[1351]:95)
at wasm::fail::ha4c23c69dfa0eea9 (wasm-function[88]:477)
at core::ops::function::FnOnce::call_once::h633718dad359559a (wasm-function[21]:22)
at wasm_bindgen_test::__rt::Context::execute::h2f669104986475eb (wasm-function[13]:291)
at __wbg_test_fail_1 (wasm-function[87]:57)
at module.exports.__wbg_apply_2ba774592c5223a7 (/home/alex/code/wasm-bindgen/target/wasm32-unknown-unknown/wbg-tmp/wasm-4a309ffe6ad80503.js:61:66)


failures:

wasm::fail

test result: FAILED. 1 passed; 1 failed; 0 ignored

error: test failed, to rerun pass '--test wasm'
```

And that's it! You've now got a test harness executing native wasm code inside
of Node.js and you can use `cargo test` as you normally would for workflows.

## Asynchronous Tests

Not all tests can execute immediately and some may need to do "blocking" work
like fetching resources and/or other bits and pieces. To accommodate this
asynchronous tests are also supported through the `futures` crate:

```rust
#[wasm_bindgen_test(async)]
fn my_test() -> impl Future<Item = (), Error = JsValue> {
// ...
}
```

The test will pass if the future resolves without panicking or returning an
error, and otherwise the test will fail.

This support is currently powered by the `wasm-bindgen-futures` crate.

## Running Tests in Headless Browsers

Add this to the root of your test crate:

```rust
wasm_bindgen_test_configure!(run_in_browser);
```

### Configuring Which Browser is Used

If one of the following environment variables is set, then the corresponding
WebDriver and browser will be used. If none of these environment variables are
set, then the `$PATH` is searched for a suitable WebDriver implementation.

#### `GECKODRIVER=path/to/geckodriver`

Use Firefox for headless browser testing, and `geckodriver` as its
WebDriver.

The `firefox` binary must be on your `$PATH`.

[Get `geckodriver` here](https://github.com/mozilla/geckodriver/releases)

#### `CHROMEDRIVER=path/to/chromedriver`

Use Chrome for headless browser testing, and `chromedriver` as its
WebDriver.

The `chrome` binary must be on your `$PATH`.

[Get `chromedriver` here](http://chromedriver.chromium.org/downloads)

#### `SAFARIDRIVER=path/to/safaridriver`

Use Safari for headless browser testing, and `safaridriver` as its
WebDriver.

This is installed by default on Mac OS. It should be able to find your Safari
installation by default.

### Debugging Headless Browser Tests

Set the `NO_HEADLESS=1` environment variable and the browser tests will not run
headless. Instead, the tests will start a local server that you can visit in
your Web browser of choices, and headless testing should not be used. You can
then use your browser's devtools to debug.
[**Read the "Testing with `wasm-bindgen-test`" section of the
guide!**](https://rustwasm.github.io/wasm-bindgen/wasm-bindgen-test/index.html)

## Components

Expand Down
10 changes: 9 additions & 1 deletion guide/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,15 @@

--------------------------------------------------------------------------------

- [Contributing](./contributing/index.md)
- [Testing with `wasm-bindgen-test`](./wasm-bindgen-test/index.md)
- [Usage](./wasm-bindgen-test/usage.md)
- [Writing Asynchronous Tests](./wasm-bindgen-test/asynchronous-tests.md)
- [Testing in Headless Browsers](./wasm-bindgen-test/browsers.md)
- [Continuous Integration](./wasm-bindgen-test/continuous-integration.md)

--------------------------------------------------------------------------------

- [Contributing to `wasm-bindgen`](./contributing/index.md)
- [Testing](./contributing/testing.md)
- [Internal Design](./contributing/design/index.md)
- [JS Objects in Rust](./contributing/design/js-objects-in-rust.md)
Expand Down
41 changes: 41 additions & 0 deletions guide/src/wasm-bindgen-test/asynchronous-tests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Writing Asynchronous Tests

Not all tests can execute immediately and some may need to do "blocking" work
like fetching resources and/or other bits and pieces. To accommodate this
asynchronous tests are also supported through the `futures` and
`wasm-bindgen-futures` crates.

To write an asynchronous test:

1. Change `#[wasm_bindgen_test]` into `#[wasm_bindgen_test(async)]`

2. Change the return type of the test function to `impl Future<Item = (), Error
= JsValue>`

The test will pass if the future resolves without panicking or returning an
error, and otherwise the test will fail.

## Example

```rust
extern crate futures;
extern crate js_sys;
extern crate wasm_bindgen_futures;

use futures::Future;
use wasm_bindgen::prelude::*;
use wasm_bindgen_futures::JsFuture;

#[wasm_bindgen_test(async)]
fn my_async_test() -> impl Future<Item = (), Error = JsValue> {
// Create a promise that is ready on the next tick of the micro task queue.
let promise = js_sys::Promise::resolve(&JsValue::from(42));

// Convert that promise into a future and make the test wait on it.
JsFuture::from(promise)
.map(|x| {
assert_eq!(x, 42);
})
.map_err(|_| unreachable!())
}
```
71 changes: 71 additions & 0 deletions guide/src/wasm-bindgen-test/browsers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Testing in Headless Browsers

## Configure Your Test Crate

Add this to the root of your test crate, e.g. `$MY_CRATE/tests/wasm.rs`:

```rust
use wasm_bindgen_test::wasm_bindgen_test_configure;

wasm_bindgen_test_configure!(run_in_browser);
```

## Configuring Which Browser is Used

> ⚡ If you are using `wasm-pack`, skip this step! Instead, use `wasm-pack test
> --chrome`, `wasm-pack test --firefox`, or `wasm-pack test --safari`.
> `wasm-pack` will automatically download and configure the appropriate
> WebDriver client for you.

If one of the following environment variables is set, then the corresponding
WebDriver and browser will be used. If none of these environment variables are
set, then the `$PATH` is searched for a suitable WebDriver implementation.

#### `GECKODRIVER=path/to/geckodriver`

Use Firefox for headless browser testing, and `geckodriver` as its
WebDriver.

The `firefox` binary must be on your `$PATH`.

[Get `geckodriver` here](https://github.com/mozilla/geckodriver/releases)

#### `CHROMEDRIVER=path/to/chromedriver`

Use Chrome for headless browser testing, and `chromedriver` as its
WebDriver.

The `chrome` binary must be on your `$PATH`.

[Get `chromedriver` here](http://chromedriver.chromium.org/downloads)

#### `SAFARIDRIVER=path/to/safaridriver`

Use Safari for headless browser testing, and `safaridriver` as its
WebDriver.

This is installed by default on Mac OS. It should be able to find your Safari
installation by default.

## Running the Tests in the Headless Browser

Once the tests are configured to run in a headless browser, executing the tests
is the same:

```bash
cargo test --target wasm32-unknown-unknown

# or, if you're using wasm-pack
wasm-pack test --headless --chrome --firefox --safari
```

### Debugging Headless Browser Tests

> If you're using `wasm-pack`, omitting the `--headless` flag will disable
> headless mode, and allow you to debug failing tests in your browser's
> devtools.

Set the `NO_HEADLESS=1` environment variable and the browser tests will not run
headless. Instead, the tests will start a local server that you can visit in
your Web browser of choices, and headless testing should not be used. You can
then use your browser's devtools to debug.
63 changes: 63 additions & 0 deletions guide/src/wasm-bindgen-test/continuous-integration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Setting Up Continuous Integration with `wasm-bindgen-test`

This page contains example configurations for running `wasm-bindgen-test`-based
tests in various CI services.

Is your favorite CI service missing? [Send us a pull
request!](https://github.com/rustwasm/wasm-bindgen)

## Travis CI

```yaml
language: rust
rust: nightly

addons:
firefox: latest
chrome: stable

install:
- rustup target add wasm32-unknown-unknown
- cargo install wasm-bindgen-cli
Copy link
Contributor

@chinedufn chinedufn Sep 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd imagine that a bunch of people will just copy this so ideally this would be a fast-enough starting point.

This can take minutes, so could it make sense to instead either:

option 1. Download / decompress a release binary and move it to either ~/.cargo/bin or /usr/local/bin

option 2. Cache cargo deps (I see that the wasm-bindgen repo's travis seems to do this?)

Happy to do either if you think it makes sense!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chinedufn totally -- good call! I think option 1 is more straight forward, and we could also have a comment mentioning something like

Download and unpack the wasm-bindgen CLI tools. Alternatively, use wasm-pack to manage wasm-bindgen binaries for you.

Thanks!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the guidance!

For posterity #824

# Install node.js with nvm.
- curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.8/install.sh | bash
- source ~/.nvm/nvm.sh
- nvm install v10.5
# Install chromedriver.
- curl --retry 5 -LO https://chromedriver.storage.googleapis.com/2.41/chromedriver_linux64.zip
- unzip chromedriver_linux64.zip
# Install geckodriver.
- curl --retry 5 -LO https://github.com/mozilla/geckodriver/releases/download/v0.21.0/geckodriver-v0.21.0-linux64.tar.gz
- tar xf geckodriver-v0.21.0-linux64.tar.gz

script:
# Test in Chrome.
- CHROMEDRIVER=$(pwd)/chromedriver cargo test --target wasm32-unknown-unknown
# Test in Firefox.
- GECKODRIVER=$(pwd)/geckodriver cargo test --target wasm32-unknown-unknown
```

## AppVeyor

```yaml
install:
- ps: Install-Product node 10
- appveyor-retry appveyor DownloadFile https://win.rustup.rs/ -FileName rustup-init.exe
- rustup-init.exe -y --default-host x86_64-pc-windows-msvc --default-toolchain nightly
- set PATH=%PATH%;C:\Users\appveyor\.cargo\bin
- rustc -V
- cargo -V
- rustup target add wasm32-unknown-unknown
- cargo install wasm-bindgen-cli

build: false

test_script:
# Test in Chrome. chromedriver is installed by default in appveyor.
- set CHROMEDRIVER=C:\Tools\WebDriver\chromedriver.exe
- cargo test --target wasm32-unknown-unknown
- set CHROMEDRIVER=
# Test in Firefox. geckodriver is also installed by default.
- set GECKODRIVER=C:\Tools\WebDriver\geckodriver.exe
- cargo test --target wasm32-unknown-unknown
```
17 changes: 17 additions & 0 deletions guide/src/wasm-bindgen-test/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Testing on `wasm32-unknown-unknown` with `wasm-bindgen-test`

The `wasm-bindgen-test` crate is an experimental test harness for Rust programs
compiled to wasm using `wasm-bindgen` and the `wasm32-unknown-unknown`
target.

## Goals

* Write tests for wasm as similar as possible to how you normally would write
`#[test]`-style unit tests for native targets.

* Run the tests with the usual `cargo test` command but with an explicit wasm
target:

```
cargo test --target wasm32-unknown-unknown
```
Loading