Gunicorn + Uvicorn workers - how does traffic get distributed?

[jesumyip]

Hi there.

I have Gunicorn managing, say, 10 Uvicorn workers i.e. gunicorn -w 10 -k uvicorn.workers.UvicornWorker.

How is incoming traffic being distributed across the 10 workers? What algorithm is used and can I control or inspect it?

For my testing, I wrote a FastAPI endpoint that sleeps for 30 seconds, and implemented a middleware that increments a counter each time a request is being processed. I then used curl in a loop to call that FastAPI a few hundred times.

@benoitc Help?

I noticed the counter is not evenly incremented across all the uvicorn workers. Some workers end up getting more traffic than others. Is this a known issue? Can someone help explain how traffic distribution works?

[jesumyip]

Nevermind. Found the solution.

[gaurav-pant-virus]

What is the solution? I am encountering same with gunicorn==20.1.0 + uvicorn

[tnusser]

@jesumyip would you mind sharing your insights instead of writing this useless response :D

[benoitc]

workers don't share any context at the moment per design. That's why the counter is not updated. the rest is documented