look into TCP fastopen and TCP_QUICKACK

[andrewrk]

Extracted from #13980.

Here's an example of using TCP fastopen to send data along with the tcp connection:

    struct msghdr mh = {
        .msg_name = (void *)sa,
        .msg_namelen = sl,
        .msg_iovlen = 2,
        .msg_iov = (struct iovec [2]){
            { .iov_base = (uint8_t[]){ ql>>8, ql }, .iov_len = 2 },
            { .iov_base = (void *)q, .iov_len = ql } }
    };
    int fd = socket(family, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0);
    if (!setsockopt(fd, IPPROTO_TCP, TCP_FASTOPEN_CONNECT, &(int){1}, sizeof(int))) {
        r = sendmsg(fd, &mh, MSG_FASTOPEN|MSG_NOSIGNAL);

TCP_QUICKACK is another option that could be enabled in setsockopt. Both options are intended to reduce latency.

This issue is to change the std lib API for making an HTTPS request to take advantage of these things, and see if we can reduce the latency of establishing a connection. For HTTP GET requests it should do the same but with the HTTP header instead of the TLS ClientHello.

[patryk4815]

TCP_FASTOPEN feature is no longer used by any major browsers due to privacy concerns. However, for internal communication within DC environments, these privacy issues may be less relevant. Probably no one is waiting for this feature :)

[rofrol]

More controversially, I suspect that Nagle’s algorithm just isn’t needed on modern systems, given the traffic and application mix, and the capabilities of the hardware we have today. In other words, TCP_NODELAY should be the default. That’s going to make some “write every byte” code slower than it would otherwise be, but those applications should be fixed anyway if we care about efficiency.

As this has gone around the internet, a number of folks have asked about TCP_QUICKACK. I don’t tend to reach for it for a few reasons, including lack of portability, and weird semantics (seriously, read the man page). The bigger problem is that TCP_QUICKACK doesn’t fix the fundamental problem of the kernel hanging on to data longer than my program wants it to. When I say write(), I mean write().

• https://brooker.co.za/blog/2024/05/09/nagle.html
• https://stackoverflow.com/questions/7286592/set-tcp-quickack-and-tcp-nodelay/34010256#34010256

[michaelortmann]

In other words, TCP_NODELAY should be the default.
Indeed, please turn off Nagle's algorithm, add TCP_NODELAY to setsockopt(fd, IPPROTO_TCP) by default.

[iamgreaser]

Not familiar with TCP_FASTOPEN but am familiar with and have strong opinions in favour of TCP_QUICKACK.

Nagle's algorithm is fairly well designed, and yet you can easily turn it off anywhere when it potentially gets in the way.

Delayed ACKs are a horrible kludge designed for a specific use case that barely ever applies (telnet over 1200 baud or something), and would be better served with Nagle's algorithm. I've seen it add 400ms of latency to a connection in a non-pathological case, and in a real-world pathological case I've seen it add 90 seconds of latency!
Disabling it is not something one can do in a portable way. But anything that can turn that mistake off on even one platform is good. Unfortunately, it's a bit of a pain on Linux, as you have to do it before every recv() or every send() or both, I forget which, safe to do it for both to be honest.

You might not end up implementing it in std, but you can bet a lot of things that I'll be doing it manually when using TCP if it isn't implemented. And having a common API to at least attempt to turn that mistake off where possible is something I will look forward to using every single time.