Description
Is your feature request related to a problem? Please describe.
This is a continuation of #4603 .
Now that #4660 is merged, there is support for basic rate-limiting via annotations.
The problem that remains is that the rate-limit is applied per-pod.
If there are for example three ingress-pods, it will effectively triple the amount of allowed request.
In case the number of ingress-pods is static, users could simply manually compensate for this.
However when autoscaling for the ingress-pods is enabled, this is not possible anymore.
Just yesterday we (again) had the case that we were hit by a DoS-Attack. Nginx scaled up to handle the traffic and started to block requests, but because of the scaling the blocking was not as effective as we hoped.
Describe the solution you'd like
I would suppose to add an additional annotation that enables the following behaviour:
The rate-limit provided by the user is divided by the amount of currently active ingress pods.
This way the total amount of allowed request always stays the same.
Yes, this will only work when incoming traffic is distributed evenly to all active nginx-pods. This will not always be the case (there could be session-presistence in the LB in front of nginx), but I suspect that this is the case most of the time.
Simple things should be simple and complicated things should be possible:
Users with a simple network-setting could use this annotation and everyone who needs more control could use nginx+ (see alternatives below).
If you are ok with this, I would implement this and open a PR.
Describe alternatives you've considered
a) Implement a kubernetes-controller that monitors the amount of running nginx-pods and automatically adjusts the ingress-objects.
-> Would surely work, but that is one more moving part that could break. And as the nginx-controller already has all the required information it feels weird to introduce an additional component.
b) Use nginx+ with zone sync
-> Would probably work, but as nginx+ is quite expensive it's not really a solution for everyone. However this might be the only viable option for users with non-trivial networking setups.