Some work in progress on running the web-platform-tests suite for WebAudio #42

orottier · 2023-11-25T19:58:50Z

From https://github.com/web-platform-tests/wpt/tree/master/webaudio Only leave 'the-audiobuffer-interface' for now. Drop the crashtests/ dir as they hang the runner

b-ma · 2023-11-26T08:53:25Z

Coo thanks! Nice that you found https://www.npmjs.com/package/wpt-runner, porting all these things manually to make them work in Node like I tried was a mess

Will have a look soon, some Node stuff can be made more idiomatic and simplified

orottier · 2023-11-26T10:35:33Z

Will have a look soon, some Node stuff can be made more idiomatic and simplified

Yes please:)

Meanwhile I'm looking for a method to collect and track the test results.
Also to not pollute the repo too much I will probably replace the wpt files with a git submodule

b-ma · 2023-11-27T08:30:26Z

Ok it's a bit more node friendly now,

Moved all the code into './bin/test-harness.mjs`
You will need to npm install again (I added some deps) and then npm run wpt

Meanwhile I'm looking for a method to collect and track the test results.

I sketched something using the programmatic API, seems to be the way to go

Also to not pollute the repo too much I will probably replace the wpt files with a git submodule

Yup maybe that would be better, I think you didn't move all the files yet isn't it?

orottier · 2023-11-28T07:29:51Z

I sketched something using the programmatic API, seems to be the way to go

Thanks, definitely beats my grep .. | uniq -c approach.

think you didn't move all the files yet isn't it?

Indeed, we make the NodeJs runtime crash for many tests and the wpt runner is not properly isolated so the whole suite terminates. I will try to see if I can add some process level isolation to the test runner but since my javascript is terrible I probably won't make it work. Alternative is to put all crashing tests into the filter method of wpt-harness.js

b-ma · 2023-11-28T07:54:29Z

Hum I see, if you have one crashing test in mind maybe you could just add it, then I can try to see what we can do?

orottier · 2023-11-28T08:05:05Z

In mobile now, but e.g. analysernode invalid constructor tests (in ctor.html file)

b-ma · 2023-11-28T10:10:27Z

ok thanks will try to have a look later today

…from other errors

b-ma · 2023-11-28T14:56:45Z

Didn't really see how fix that in the test runner, seems more like a napi issue to me, so I fixed the issue in the lib itself (which is anyway what we want I guess :)

I also discriminated all errors of kind threw "Error" instead of EcmaScript error TypeError. from "real" errors in the output. So the test still not pass but at least we know this is not a behaviour problem. (I really thought a lot on how to fix this as at first sight it seems like an easy win, but I'm afraid the only solution is to rewrap absolutely everything in JS with try / catch to properly handle exotic error types, doable but so boring...)

All in all, at first sight:

Good new is that many issues comes from the bindings and not from the rust crate.
Sad new is that many issues comes from the bindings and not from the rust crate.

(I really like the fact that the ScriptProcessorNode is deprecated but that some tests rely on it :)

orottier · 2023-11-28T16:57:41Z

Thanks for the additional work. Adding a different result type for "expected error ... but got error .." is very useful.

The wpt runner still crashes a lot for stupid reasons (missing bindings) but I also discovered a real bug, nice.

orottier · 2023-11-29T08:06:28Z

Slowly making progress, I managed to get into the the-audio-node-interface directory

I need these changes in the wpt repo, I will make a fork for that later

	deleted:    webaudio/the-audio-api/the-audiocontext-interface/audiocontext-not-fully-active.html
	deleted:    webaudio/the-audio-api/the-audiocontext-interface/audiocontextoptions.html
	deleted:    webaudio/the-audio-api/the-audiocontext-interface/constructor-allowed-to-start.html
	deleted:    webaudio/the-audio-api/the-audiocontext-interface/suspend-with-navigation.html
	deleted:    webaudio/the-audio-api/the-mediaelementaudiosourcenode-interface/cors-check.https.html
	deleted:    webaudio/the-audio-api/the-mediaelementaudiosourcenode-interface/no-cors.https.html

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	webaudio/resources/buffer-loader.js

That new file is created with cp wpt/webaudio/js/buffer-loader.js wpt/webaudio/resources.

I will create a tracking issue with all issues that make the runner crash (and hence why I deleted these 6 files)

orottier · 2023-11-30T08:18:02Z

Alright, I got the full suite to run.
For the rust project, you need the feature/prevent-wpt-crashes branch
For wpt, you need the https://github.com/orottier/wpt fork
Also I'm pushing some more stubs to this project

orottier · 2023-11-30T16:33:17Z

For wpt @ web-platform-tests/wpt@83f318a
node bindings @ ce30937
rust impl @ orottier/web-audio-api-rs@0dab4c6

  RESULTS:
  - # pass: 3020
  - # fail: 1293
  - # type error issues: 230
  
files ignored: 21

b-ma · 2023-12-01T06:42:20Z

Cool, nice work!

Few question/remarks:

About the files you deleted in your wtp fork, it wasn't possible to just filter them in the wpt-harness.mjs script?
For the listener, we agree that the way you did it will break the kind of lazy instantiation we have in the rust side? I do not mean this should be fixed right away if this is the case, but maybe we should create a new issue to fix that

Let me know if you need some help on some points and/or when you think this is ready to merge

b-ma · 2023-12-01T07:10:31Z

src/offline_audio_context.rs

+    let obj = match napi_obj.0.as_ref() {
+        Some(v) => v,
+        None => return ctx.env.create_double(0.),
+    };


I was a bit surprised about this change at first, but I guess it prevents crashing when we do something like this?

const result = await offlineContext.startRendering(); const now = offlineContext.currentTime

Seems that all this Option stuff was required to work around:

pub fn start_rendering_sync(self) -> AudioBuffer

Did you think it could be resolved at the rust level to have something like this instead:

pub fn start_rendering_sync(&self) -> AudioBuffer

Yes definitely, let's change that API. And also implement https://webaudio.github.io/web-audio-api/#dom-offlineaudiocontext-suspend because it is used so much in the wpt.

orottier · 2023-12-02T08:07:16Z

About the files you deleted in your wtp fork, it wasn't possible to just filter them in the wpt-harness.mjs script?

Yes that would probably be better. My intend is to get rid of the fork soon, but we also have this issue of the missing resource file. I think that needs patching in wpt-runner repo.

For the listener, we agree that the way you did it will break the kind of lazy instantiation we have in the rust side? I do not mean this should be fixed right away if this is the case, but maybe we should create a new issue to fix that

Hum right, this has a large performance impact. I could probably use your help here. Is there a way to lazily return the Listener on request? Also let's move the Listener bindings to a separate PR, right?

…into feature/wpt-harness

orottier · 2023-12-06T07:38:21Z

New results with webaudio-rs v0.38 en new templating:

  RESULTS:
  - # pass: 3285 (+85)
  - # fail: 1444 (+97)
  - # type error issues: 235 (=)

files ignored: 9 (=)

Previous results #42 (comment)

…dioContext

Attempting to get OfflineAudioContext.suspend to work

to prevent a wpt crash

You may need to run `git submodule sync --recursive`

orottier · 2023-12-27T09:04:54Z

New results with suspend/resume and AudioListener.setPosition

  RESULTS:
  - # pass: 3386 (+101)
  - # fail: 1380 (-64)
  - # type error issues: 235 (=)

files ignored: 8 (-1)

Previous results #42 (comment)

b-ma · 2023-12-27T17:28:26Z

src/audio_listener.rs

+    //
+    // When any of the positionX, positionY, and positionZ AudioParams for this AudioListener have
+    // an automation curve set using setValueCurveAtTime() at the time this method is called, a
+    // NotSupportedError MUST be thrown.


I think this is actually ok as the error will be thrown by the set_value call. The check is done in the AudioParam.

But this is also something that should be fixed on the rust side at some point because the panic would occur in the render thread which is not good...

b-ma · 2023-12-27T17:34:26Z

Hey, really good job thanks! It's very nice to have such move forward on that point.

I will just add a small fix on AudioParam tomorrow before merging. Then I will refactor the JS code to make it more clean as described in #53, I think it will help to continue on that path more simply

b-ma · 2023-12-28T11:11:07Z

Ok to merge on my side! Good for you?

orottier · 2023-12-28T11:12:28Z

All good 👍

orottier · 2023-12-28T12:54:58Z

New results with all audioparam attributes and OfflineAudioContext::state

  RESULTS:
  - # pass: 3413 (+27)
  - # fail: 1353 (-27)
  - # type error issues: 235 (=)

files ignored: 8 (=)

Previous results #42 (comment)

orottier added 2 commits November 25, 2023 19:40

Copy over some web-platform-tests

1acdd32

From https://github.com/web-platform-tests/wpt/tree/master/webaudio Only leave 'the-audiobuffer-interface' for now. Drop the crashtests/ dir as they hang the runner

Run the WPT webaudio test harness for our bindings

b98fd01

b-ma added 3 commits November 27, 2023 08:57

refactor: move wpt as npm script

66fdf55

chore: use programmatic API to run wpt

ced86f1

chore: clean wpt files

7f9cd2e

chore: build in debug mode when running wpt

b27bc9c

b-ma added 2 commits November 28, 2023 15:34

Merge branch 'main' into feature/wpt-harness

e8835d6

test: add analyser node wpt test + disciminate errors of error types …

0eb0b7e

…from other errors

test: add some options for wpt tests

8ff8d03

orottier added 3 commits November 28, 2023 17:58

Avoid some wpt crashes by providing stub implementations

acfbb2e

Replace wpt webaudio suite with git submodule

99bfa66

Skip crashtests and resources in web platform tests

81e52ed

Add more stubs to prevent the wpt from crashing

ce30937

orottier added 2 commits November 30, 2023 21:02

Add bindings for AudioListener

edad866

Revert unrelated changes to iir-filter example

072baef

b-ma reviewed Dec 1, 2023

View reviewed changes

orottier added 4 commits December 3, 2023 11:48

Merge remote-tracking branch 'origin/refactor/offline-audio-context' …

4dddcf5

…into feature/wpt-harness

Revert other changes for OfflineAudioContext being consumed

e675723

Merge remote-tracking branch 'origin/main' into feature/wpt-harness

686c466

Remove printlns

babd5bd

orottier and others added 8 commits December 7, 2023 08:51

Attempting to get OfflineAudioContext.suspend to work

4374dd9

make callback run without crashing

1ed0c50

feat: implement async startRendering, susped and resume for OfflineAu…

8f4d454

…dioContext

Merge branch 'main' into feature/offline-context-suspend

3461cd9

refactor: cleaning

17770ef

fmt

56ff66e

OfflineAudioContext renamed suspend_at -> suspend

df44e9f

Merge pull request #50 from ircam-ismm/feature/offline-context-suspend

a1d7a3e

Attempting to get OfflineAudioContext.suspend to work

orottier mentioned this pull request Dec 27, 2023

Attempting to get OfflineAudioContext.suspend to work #50

Merged

orottier added 2 commits December 27, 2023 09:38

Implement deprecated AudioListener setPosition, setOrientation

c24af51

to prevent a wpt crash

Update the wpt submodule to my own fork for now

ca9014e

You may need to run `git submodule sync --recursive`

Upgrade to web-audio-api-rs 0.39 and reinstate lto setting

2633aed

orottier marked this pull request as ready for review December 27, 2023 09:13

b-ma reviewed Dec 27, 2023

View reviewed changes

b-ma added 3 commits December 28, 2023 09:11

feat: implement all audio param attributes

edb3441

fix: typo

fd4d710

fix: expose OfflineAudioContext::state

1a3e918

b-ma merged commit b8d668b into main Dec 28, 2023

b-ma deleted the feature/wpt-harness branch December 28, 2023 11:13

orottier mentioned this pull request Dec 28, 2023

The web-platform-tests tracking issue #57

Open

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Some work in progress on running the web-platform-tests suite for WebAudio #42

Some work in progress on running the web-platform-tests suite for WebAudio #42

orottier commented Nov 25, 2023 •

edited

Loading

b-ma commented Nov 26, 2023

orottier commented Nov 26, 2023

b-ma commented Nov 27, 2023

orottier commented Nov 28, 2023

b-ma commented Nov 28, 2023 •

edited

Loading

orottier commented Nov 28, 2023

b-ma commented Nov 28, 2023

b-ma commented Nov 28, 2023 •

edited

Loading

orottier commented Nov 28, 2023 •

edited

Loading

orottier commented Nov 29, 2023

orottier commented Nov 30, 2023

orottier commented Nov 30, 2023 •

edited

Loading

b-ma commented Dec 1, 2023 •

edited

Loading

b-ma Dec 1, 2023

orottier Dec 2, 2023

orottier commented Dec 2, 2023

orottier commented Dec 6, 2023

orottier commented Dec 27, 2023

b-ma Dec 27, 2023

b-ma commented Dec 27, 2023

b-ma commented Dec 28, 2023

orottier commented Dec 28, 2023

orottier commented Dec 28, 2023 •

edited

Loading

Some work in progress on running the web-platform-tests suite for WebAudio #42

Some work in progress on running the web-platform-tests suite for WebAudio #42

Conversation

orottier commented Nov 25, 2023 • edited Loading

b-ma commented Nov 26, 2023

orottier commented Nov 26, 2023

b-ma commented Nov 27, 2023

orottier commented Nov 28, 2023

b-ma commented Nov 28, 2023 • edited Loading

orottier commented Nov 28, 2023

b-ma commented Nov 28, 2023

b-ma commented Nov 28, 2023 • edited Loading

orottier commented Nov 28, 2023 • edited Loading

orottier commented Nov 29, 2023

orottier commented Nov 30, 2023

orottier commented Nov 30, 2023 • edited Loading

b-ma commented Dec 1, 2023 • edited Loading

b-ma Dec 1, 2023

Choose a reason for hiding this comment

orottier Dec 2, 2023

Choose a reason for hiding this comment

orottier commented Dec 2, 2023

orottier commented Dec 6, 2023

orottier commented Dec 27, 2023

b-ma Dec 27, 2023

Choose a reason for hiding this comment

b-ma commented Dec 27, 2023

b-ma commented Dec 28, 2023

orottier commented Dec 28, 2023

orottier commented Dec 28, 2023 • edited Loading

orottier commented Nov 25, 2023 •

edited

Loading

b-ma commented Nov 28, 2023 •

edited

Loading

b-ma commented Nov 28, 2023 •

edited

Loading

orottier commented Nov 28, 2023 •

edited

Loading

orottier commented Nov 30, 2023 •

edited

Loading

b-ma commented Dec 1, 2023 •

edited

Loading

orottier commented Dec 28, 2023 •

edited

Loading