Payload masking is ridiculously slow

[u3shit]

I'm trying to transfer a bigger amount of data (~50-100MB, in 1MB chunks) through a binary[1] websocket, but it's ridiculously slow (it can take 8-10 seconds to transfer 50MB of data on localhost, while both my client and server uses 100% CPU). After a bit of profiling, it turned out that more than 99% of the total runtime is spent here:

protocol-websocket/lib/protocol/websocket/frame.rb

Lines 105 to 107 in 6280c19

	for i in 0...data.bytesize do
	@payload << (data.getbyte(i) ^ mask.getbyte(i % 4))
	end

Collecting the relevant parts into a standalone ruby file:

require 'securerandom'

mask = SecureRandom.bytes(4)
data = ' ' * (1024*1024*50)

@payload = String.new.b
for i in 0...data.bytesize do
  @payload << (data.getbyte(i) ^ mask.getbyte(i % 4))
end

Running this code takes about 8.6s on my machine (the initialization part is about 0.09s, ruby 3.1.2, amd64, gentoo linux).

Doing things inplace, and no modulo is a bit faster, about 7.3s:

(0...data.bytesize).each do |i|
  data.setbyte(i, data.getbyte(i) ^ mask.getbyte(i & 3))
end

(Interestingly Range#each is faster than the for loop.) Is it possible to do something about this other than writing a C extension?
For the time being, I've worked around the problem by changint the mask: true to false here: https://github.com/socketry/async-websocket/blob/a63151d0759edde5ca5cd9ffa0414d9f2e295ee8/lib/async/websocket/client.rb#L39 which works on my development setup, but it's AFAIK a violation of the websocket spec, so it could break with proxies and whatever.

[1]: I'm using a handler class like this, because transferring binary data in json is not fun:

class BinaryWebSocket < Async::WebSocket::Connection
  def parse x; x; end
  def dump x; x; end
end

[ioquatix]

Damn, maybe we can make a C extension to handle this.

[ioquatix]

ruby/ruby#5893 should provide what we need.

[ioquatix]

I believe using HTTPS or HTTP/2 can also avoid this problem.

[u3shit]

I wanted to write that I'm already using HTTPS, but then I realized that I probably made the above measurements in my devenv which only used plain HTTP.
But why would HTTPS matter? Looking at the RFC6455, it doesn't mention anything about not using masking with secure connections (I think skipping masking could still cause problems with TLS terminating proxies, but that might be a niche usecase... on the other hand, we're talking about a ruby client that can open raw sockets if it wants, not a javascript code running in a sandbox where masking actually makes sense.)

[ioquatix]

@u3shit I have proposed #16 to improve the performance.

If you have time, do you mind checking the performance? i.e. can you run some benchmarks and see if it's good or not?

We cannot introduce this until IO::Buffer is not experimental.

[u3shit]

I've retried one of my old examples I knew that was slow (transferring 50MB binary data in chunks). Actually I had to replace it to send random data instead of zero bytes because all zero data was no longer slow, anyhow:

• ruby 3.2.2, upstream protocol-websocket-0.11.0 with my workaround from OP: 0.949s
• ruby 3.2.2, upstream protocol-websocket-0.11.0: 6.633s
• ruby 3.2.2, io-buffer-masking branch: 1.027s

Then I figured out how to disable compression (I think they weren't enabled by default previously), with that the numbers are:

• ruby 3.2.2, upstream protocol-websocket-0.11.0 with my workaround from OP: 0.120s
• ruby 3.2.2, upstream protocol-websocket-0.11.0: 5.891s
• ruby 3.2.2, io-buffer-masking branch: 0.217s

Note that this was done on localhost, on a real internet connection compression probably makes more sense (not with the random data, thought).

I guess this is an alright change, given that my workaround probably doesn't even conform to the websocket spec (I still find that masking a bit superfluous, it makes sense when you want to protect against malicious JS code trying to exploit buggy proxies, but in Ruby you can open a raw TCP socket and write whatever you want into it, but I guess that's my problem...)

[ioquatix]

I agree that masking is annoying - slow and clunky for questionable trade offs.

However, as you said, it might be against the spec if it's not used.

In any case, with the PR, the overhead is greatly reduced, which is awesome. Thanks for those numbers.

[ioquatix]

I've released this with a conservative feature guard in v0.12.0

[ioquatix]

Just for reference, my own direct benchmark:

> ruby mask_xor.rb 
Warming up --------------------------------------
          IO::Buffer   127.000  i/100ms
              String     1.000  i/100ms
Calculating -------------------------------------
          IO::Buffer      1.294k (± 1.2%) i/s -      6.477k in   5.007365s
              String     11.643  (± 0.0%) i/s -     59.000  in   5.068373s

Comparison:
          IO::Buffer:     1293.7 i/s
              String:       11.6 i/s - 111.11x  slower