uWebsockets publish dropping messages

[rpgehlot]

Hi @alexhultman ,
Thanks for this great and efficient library which i have to started to use recently in place of socket.io which was giving us horrible performance in production.
Basically we are using it in context of a trading platform which is in house to our firm. There we get around ~400 msg/sec of ~2KB with ~60 connected websocket clients. We are using uWebsockets pub-sub with maxBackPressure setting of 6KB.
Everything was fine, but we noticed some messages are being dropped. I understand its happening because of built up backpressure because of which it drops messages for slow client or either drops their connection.
Also i tried using ws.send method and manage backpressure using ws.getBufferedAmount() and implement my own pub-sub over it but obviously it was much slower and useless.

I really want to use your library becuase its way efficient than others, but if you can help me to fix this problem of message drop, that would be very grateful of you.

[hst-m]

I understand its happening because of built up backpressure because of which it drops messages

Are you saying you'd prefer it drop the connection rather than drop messages? Docs need to be updated, there is new option closeOnBackpressureLimit to drop connection rather than drop message. If you don't want it doing either you should raise limit

Hi, https://github.com/uNetworking/uWebSockets/releases/tag/v18.22.0 did introduce closeOnBackpressureLimit but it looks like that option was never wrapped here.

@rpgehlot Can you test the very latest binaries? https://github.com/uNetworking/uWebSockets.js/tree/binaries

they should have closeOnBackpressureLimit option

[rpgehlot]

Thanks for your prompt reply, but i needed centos 7 binary and i actually dont have root access so can't build that bin right now. will get it built and try out it this option.

[rpgehlot]

Hi, I tried your new binary it seems to work, but i have a question if i increase the maxBackpressure to 11MB for e.g, how would it affect the latency of packets reaching the browsers ? would the perf suffer because of high maxbackpressure ?

[rpgehlot]

I am seeing poor performance with maxBackpressure=11MB and closeOnBackpressureLimit : true and no drops, and low latency with maxBackpressure=6KB and closeOnBackpressureLimit:false but with message drops, i guess there is no middle ground. or is there ?

11MB backpressure sounds like a lot if you care about latency. If latency is a concern, set backpressure low. But either way you cannot just shove messages like there's no tomorrow - if you constantly end up with messages stuck in backpressure then you're sending too quick.

[rpgehlot]

So, if we can't control that incoming rate, would some kind of rate limiter would be helpful which can control outward messages sent per sec to each client ? and is there a possibility we can use such a thing with publish ?

[hst-m]

i guess there is no middle ground. or is there ?

11MB is 2000 times larger than 6KB, the middle ground would be all the values in between

we get around ~400 msg/sec of ~2KB with ~60 connected websocket clients

each message is 2KB? so you are publishing 800KB per second to 60 clients? or is that 2KB per sec, 5 bytes per message?

[rpgehlot]

thats the worst case figures i have mentioned, message rate is actually variable depending on the trading volume. So yes it would amount to 800KB per sec for a single client and we have 60 such clients. most of the messages are 2KB but initial first message amounts to a little bigger value upon connection start, so thats why i used 11MB. for case when a single message is > than maxbackpressure, if i use a smaller value of 6KB then it actually closes the connection at start itself and stuck in a infinite loop since i used closeOnBackpressureLimit : true

[uasan]

I think you need to pause before sending the next message if there is backpressure on the socket

if (socket.write(message) === false) {
  await socket.onDrain();
}

[hst-m]

So you tried 6KB which is smaller than a single message, and you tried 11MB which is 14 seconds of 800KB/sec. Have you considered trying anything between those 2 values?

800KB/sec seems like a lot, are you sure these are efficiently packed messages? Binary format etc?

[rpgehlot]

No i haven't tried anything b/w. I encode json messages to arraybuffer(binary message) as that is preferred in uWS (from docs).
Currenlty i am sending all messages using
uApp.publish(<message_room>, binarybuffer, true, true) and it should do compression as last flag is true.
I am thinking to break the bigger message in chunks size smaller than maxbackpressure, and send using ws.send if there is no backpressure else wait for drain to send the next chunk. So a combination of ws.send and ws.publish but publishing only when whole chunk is sent. (as there io no order guarantee b/w publish and send).
This hopefully should give me the perf boost using smaller maxbackpressure and also to avoid the problem of connection close on first message.