Audio not working within the html5 client on Firefox

[pchihak]

Describe the bug
When accessing the xpra html5 client in Firefox audio will not work and I am unable to load any audio codecs despite the server claiming to support them.

To Reproduce
Steps to reproduce the behavior:

1. xpra start --daemon=no -d gstreamer,sound --start-child='vlc'
2. Connect to the html5 client via firefox https:///index.html specifically choosing the mp3 audio codec.
3. Open any mp3 file within vlc and the file will successfully load and play however no audio will reach the browser.

System Information (please complete the following information):

• Server OS: Debian Bullseye
• Client OS: Debian Bullseye
• Xpra Server Version: xpra v4.3.4-r0 installed from the xpra.org packages
• Xpra Client Version: latest version of xpra-html5 installed from xpra.org packages. Presumably the same as the xpra server version?
• Firefox Version: 103.0.2
• Chrome Version: 104.0.5112.102 (Official Build) (64-bit) (cohort: Stable)

Additional context
When I connect to the html5 client from within Firefox and specifically try to force the mp3 codec I receive an error audio codec mp3+mpeg4 is not supported by the server and then the html5-client falls back to the aac+mpeg4 codec. This also happens with every other audio codec that I attempt to use. Additionally when the html5 client and server are negotiating what codec to use the server does send the array of supported codecs and mp3 is within that array however as the client agrees mp3+mpeg4 is not.

audio codecs supported by the server: Array(16) [ "opus", "flac", "wavpack", "wav+lz4", "wav", "opus+ogg", "vorbis+mka", "vorbis+ogg", "vorbis", "mp3", … ] 0: "opus" ​ 1: "flac" ​ 2: "wavpack" ​ 3: "wav+lz4" ​ 4: "wav" ​ 5: "opus+ogg" ​ 6: "vorbis+mka" ​ 7: "vorbis+ogg" ​ 8: "vorbis" ​ 9: "mp3" ​ 10: "mp3+id3v2" ​ 11: "flac+ogg" ​ 12: "aac+mpeg4" ​ 13: "speex+ogg" ​ 14: "vorbis" ​ 15: "opus+mka"

However when I connect to the xpra server using xpra attach tcp://<IP ADDR>:14500 --speaker-codec=mp3 then I can get audio working without issue.

While writing this I went ahead and also tried loading the html5 client within Chrome instead of Firefox and the audio did work however I then run into lots of freezing, and the application becomes unresponsive which is probably its own ticket so I won't get into that much here until I can get a better idea what is going on with that.

[totaam]

IIRC, with the html5 client we detect what is available then also apply a blacklist for some of these options.
(and perhaps this list should be updated)

I'm quite certain that it at least used to work with Firefox: #23 (comment)
(no idea what has changed since as I don't use Firefox much)

The proper solution is likely to be #64 and it should be quite well supported by browsers nowadays.

[totaam]

Although there is a noticeable delay, audio still does work with Firefox as long as you have all the gstreamer elements installed.

audio codec mp3+mpeg4 is not supported by the server

That's your problem.

Here is the pipeline it ends up choosing for me:

pulsesrc name="src" ! \
timestamp name=timestamp !\
queue name=queue min-threshold-time=0 max-size-buffers=0 max-size-bytes=0 max-size-time=50000000000000 leaky=2 !\
audioconvert !\
removesilence !\
audioconvert !\
audioresample !\
volume name=volume volume=1.0 !\
avenc_aac perfect-timestamp=1 !\
mp4mux faststart=1 streamable=1 fragment-duration=20 presentation-time=0 !\
appsink name=sink emit-signals=true max-buffers=10 drop=true sync=false async=false qos=false

And the server shows:

2023-03-16 21:10:20,094 audio capture using 'aac' audio codec
2023-03-16 21:10:20,358 audio capture using 'iso fmp4' container format

So you need avenc_aac and mp4mux.
There 2 other options:

• avenc_aac + mp4mux
• Xpra-org/xpra@e781977 just added voaacenc + mp4mux, which I am unable to test.

---

As for the hangs, I believe you must have hit: #217 or #212

[totaam]

Works for me.