Support Hybi10 binaryType

[ebidel]

Just landed in Chromium: http://code.google.com/p/chromium/issues/detail?id=93652

Allows sending Blob or ArrayBuffer (binary data).

[einaros]

The reason why I didn't implement this part of the hybi draft is simply that the other transports would struggle with it. It'll be better to add this when the draft stabilizes, and more browsers support it.

[ebidel]

By other transports do you mean other protocol versions? Regardless of whether or not the current spec is stable, expect binary support to stay.

[ebidel]

Is this fixed with v0.85? I see binary support in the release notes.

[einaros]

Well it's in the parser at least, but it has yet to be connected to anything meaningful.

Now is a good time to ask: What binary data do you actually want to submit?

[ebidel]

Use cases are many. Here are 2 off the top of my head:

• stream audio to connected clients
• realtime screen collaboration tool: stream the binary images across the wire, render to canvas.

https://github.com/Worlize/WebSocket-Node seems to support binary transfer.

[einaros]

Supporting binary transfer is, as I said, no big deal. I just struggle to see real use cases, seeing as js on the client has no binary data type.

Stream audio: How would you decode the binary efficiently, and assign them to the float arrays used by the sound apis (which are still not standard)?

Stream binary images: Canvas painting would either (a) be a video element which is drawn to a canvas, (b) an image which is drawn to a canvas, (c) rgba number values set to the bitmapdata of the canvas. Of the three, neither could really use binary data sent via websockets. And (c), being the closest to even remotely possible, is insanely slow.

Edit: I see that Chrome at least support receiving arraybuffers now, which was ironically your opening statement for this issue, so I should have caught that. Anyhoo, the audio bit should be doable as you say, and probably canvas as well if it now has a setter for the typed arrays.

[ebidel]

js on the client has no binary data type.

Yep. They're many new types in JS to work with binary data: Typed Arrays, ArrayBuffer, Blob, File.
They're used all over the place now (webgl, audio apis, file apis, xhr2). I presented an entire section on this at Google IO:
http://www.htmlfivewow.com/slide11

Decoding audio:

The Web Audio API for example (now supported in Chrome and WebKit nightlies) has methods for decoding audio:
https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html#AudioContext-section

You send it an ArrayBuffer.

streaming binary images

I was thinking more like a remote desktop client. Images are rapidly sent over a websocket (without the need to base64 encode, etc.) and rendered onto a canvas. c.) could also work, since the updated protocol allows passing ArrayBuffers.

I believe they're swamping out canvas's CanvasPixelArray to use ArrayBuffer instead. Regardless, you'll
be able to .getBlob() from the canvas and create an image file from it:
http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#dom-canvas-toblob

[einaros]

I was familiar with the floating point types, but not blob / file and alike. Granted those, and also canvas' push for ArrayBuffer, it makes sense to complete this implementation sooner rather than later.

I'll see if I can find time to do so later this week.

[ebidel]

You da man! Thank you!

[mmarcon]

Was this ever implemented? I am curious to play with images on canvas to be sent to the server for heavy processing and ArrayBuffer over socket.io seems like a pretty solution.

[einaros]

I implemented this partially in socket.io, but fully in 'ws', my up-to-date websocket client/server for node. See https://github.com/einaros/ws

[ebidel]

What is missing on the socket.io side of things?

[einaros]

• Support across the other transports.
• Merge of my pullreq which uses ws over my old parsers in socket.io, OR:
• Switch to engine.io+websocket.io and exposure of ws' binary features through websocket.io.

I added more binary samples to ws, by the way.