Add github action for testing #65
Conversation
Just checking to see how far this gets currently or if we need to disable webgpu tests.
brendandahl
changed the title
brendandahl
changed the title
| runs-on: macos-latest | ||
| strategy: | ||
| matrix: | ||
| # TODO enable safari and firefox |
There was a problem hiding this comment.
side comment: I've seen that there are some resource limitations with macos so I've split the tasks in JetStream to be able to run on linux (though these are slightly slower machines).
https://github.com/WebKit/JetStream/blob/main/.github/workflows/test.yml
tests/run-end2end.mjs
Outdated
| async function test() { | ||
| try { | ||
| await driver.manage().setTimeouts({ script: 60000 }); | ||
| await driver.manage().setTimeouts({ script: 20 * 60000 }); |
There was a problem hiding this comment.
very small nit: Could you make 60000 a const ONE_MINUTE_IN_MS = 60000 or so, to make this a somewhat less magic number?
Also, at a high-level: 20 minutes is a really long time out, do we have a lighter workload?
There was a problem hiding this comment.
For a lighter workload, we could run one benchmark from each of the frameworks and backend types. e.g. 1 transformerjs w/ wasm/gpu, 1 tflite w/ wasm/gpu. They're all basically the same code so that'd probably catch most problems.
There was a problem hiding this comment.
I decided it would be a bit better to get more coverage and do all wasm tests and 2 gpu tests (1 from tflite and transformers). The wasm tests tests on CI seem relatively quick.
4 participants
The full e2e tests are very slow, but they pass with an increased timeout.
Enabled for chrome only for now.
Fixes #60