option to use SO_REUSEPORT ?

[benoitc]

Linux 3.9 added the socket option SO_REUSEPORT which is also available in freebsd. Maybe we could have an option to use it or if present, use it. One advantage I see in it is that it would improve the way connections are distributed across processes.

See the post on LWN for more information:

https://lwn.net/Articles/542629/

In which we can read the difference with using SO_REUSEADDR as we do now:

The problem with this technique, as Tom pointed out, is that when multiple threads are waiting in the accept() call, wake-ups are not fair, so that, under high load, incoming connections may be distributed across threads in a very unbalanced fashion. At Google, they have seen a factor-of-three difference between the thread accepting the most connections and the thread accepting the fewest connections; that sort of imbalance can lead to underutilization of CPU cores. By contrast, the SO_REUSEPORT implementation distributes connections evenly across all of the threads (or processes) that are blocked in accept() on the same port.

Thoughts?

[stefanor]

Besides the load balancing advantage, I can also imagine that this would enable zero-downtime deploys with Gunicorn:

1. Spin up new instances, running the new code. They start taking some connections.
2. Ask the old ones to stop accepting(), finish any existing connections, and shut down.

[benoitc]

@stefanor this can already be done using the USR2 signal :) But it may ease the process indeed.

[stefanor]

Oh, did not know that :) it looks like that may even do what I need (with some hacking).

[end of bug hijacking]

[benoitc]

Maybe that could help you:

http://docs.gunicorn.org/en/latest/signals.html#upgrading-to-a-new-binary-on-the-fly

On Fri, Sep 6, 2013 at 1:50 PM, Stefano Rivera notifications@github.comwrote:

Oh, did not know that :) it looks like that may even do what I need (with
some hacking).

[end of bug hijacking]

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/598#issuecomment-23934724
.

[tilgovi]

Reasonable and shouldn't be too hard.

[georgexsh]

If I didn't misunderstand the article, with this feature, there is no need to add accept mutex to solve the thundering herd problem of gunicorn.

[benoitc]

@georgexsh hrmmmm good call. it coul dbe one of the supported scenario. Need to find a way to detect if the feature is supported or not.

[asvetlov]

From my understanding the only way is to call sock.setsockopt(SOL_SOCKET,
SO_REUSEPORT, 1) and catch exception on unsupported environment.

On Wed, Aug 20, 2014 at 2:29 PM, Benoit Chesneau notifications@github.com
wrote:

@georgexsh https://github.com/georgexsh hrmmmm good call. it coul dbe
one of the supported scenario. Need to find a way to detect if the feature
is supported or not.

—
Reply to this email directly or view it on GitHub
#598 (comment).

Thanks,
Andrew Svetlov

[dPeS]

+1 waiting for this, how can I help?

[berkerpeksag]

@dPeS you can send a pull request :)

[dropwhile]

I've been fiddling around with a branch that attempts to add support for so_reuseport. Long way to go to get it working yet though.

[smilliken]

Here's a simple commit that enables SO_REUSEPORT: smilliken@9e66c7a

This works on production systems running Python 2.7.9 and various versions of Linux 3.x (before and after 3.9 in which the feature was added). It likely breaks with an AttributeError on earlier versions of Python that don't define socket.SO_REUSEPORT (untested).

I'm using it for zero-downtime deploys, enabling multiple parent gunicorn processes to run concurrently during a transition, while the old processes finish their requests and exit gracefully. I couldn't use USR2 signal as suggested, since I need to handle environmental changes as well. Specifically, the new gunicorn processes need to be spawned from a new environment to have the correct state. I imagine it would be useful for other environmental changes, like process owner or cgroups.

[tilgovi]

@smilliken it would be great if you could open a PR from a branch with that commit. Maybe we should add an option in gunicorn/config.py. It could test hasattr on the socket module and reject the option if there is no support.