Skip to content

Commit

Permalink
Problem: Windows 7 TCP slow start
Browse files Browse the repository at this point in the history
See issue zeromq#1608.

This is an old issue with Windows 7. The effect is that we see a latency
ramp on the first 500 messages.

* The ramp is unaffected by message size.
* Sleeping up to 100msec between sends has no effect except to switch
  off ZeroMQ batching so making the ramp more visible.
* After 500 messages, latency falls back down to ~10-40 usec.
* Over inproc:// the ramp happens when we use the signaler class.
* Client-server over inproc:// does not show the ramp.
* Client-server over tcp:// shows a similar ramp.

We know that the signaller is using TCP on Windows. We can 'prime' the
connection by doing 500 dummy sends. This potentially causes new sockets
to be delayed on creation, which is not a good solution.

Note that the signaller sends zero-byte messages. This may also be
confusing TCP.

Solution: flood the receive buffer when creating a new FD pair; send a
1M buffer and discard it.

** THIS IS SPECULATIVE **
  • Loading branch information
hintjens committed Oct 30, 2015
1 parent f38c11c commit aec874c
Showing 1 changed file with 20 additions and 0 deletions.
20 changes: 20 additions & 0 deletions src/signaler.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -511,6 +511,26 @@ int zmq::signaler_t::make_fdpair (fd_t *r_, fd_t *w_)
if (rc != SOCKET_ERROR)
*r_ = accept (listener, NULL, NULL);

// Send/receive large chunk to work around TCP slow start
// This code is a speculative workaround for #1608
if (*r_ != INVALID_SOCKET) {
size_t dummy_size = 1024 * 1024; // 1M to overload default receive buffer
unsigned char *dummy = (unsigned char *) malloc(dummy_size);
int still_to_send = (int) dummy_size;
int still_to_recv = (int) dummy_size;
while (still_to_send && still_to_recv) {
int nbytes;
if (still_to_send > 0) {
nbytes = ::send (*w_, (char *)(dummy + dummy_size - still_to_send), still_to_send, 0);
wsa_assert (nbytes != SOCKET_ERROR);
still_to_send -= nbytes;
}
nbytes = ::recv (*r_, (char *)(dummy + dummy_size - still_to_recv), still_to_recv, 0);
wsa_assert (nbytes != SOCKET_ERROR);
still_to_recv -= nbytes;
}
free (dummy);
}
// Save errno if error occurred in bind/listen/connect/accept.
int saved_errno = 0;
if (*r_ == INVALID_SOCKET)
Expand Down

0 comments on commit aec874c

Please sign in to comment.