Pod to Pod Bandwidth Reservation

[mudverma]

Proposal/Question:

For Bandwidth sensitive telco and or video/audio streaming applications we need to block bandwidth between pods. This is specially needed in service chaining scenarios.

Currently calico supports local traffic shaping (bandwidth reservation) per pod. What we want is bandwidth reservation between pods.

Can this be achieved with current implementation? If not, we are considering adding such a feature.

We want to take community opinion on this.

[Levovar]

yeah, the holy grail of universal traffic shaping :)
I do agree with you that this is required, and we were also internally considering this for a time now.
currently it is not possible

I don't have an exact design for this, but such a solution should adhere to some basic requirements:

• it should be a universal solution, e.g. work with at least all dyanmic CNI backends (SR-IOV, MACVLAN, IPVLAN)
• because of this the shaping rules (I guess some iptables magic) probably need to be provisioned inside the Pod's network namespace
• need to be API driven. I think this shouldn't be added to the existing DANM API, but would be better controlled via a new CRD, with a new independent Controller applying the changes (similarly to a NetworkPolicy Controller)
  as such htis functionality would better belong to our utility repo, rather than the core (https://github.com/nokia/danm-utils)

[mudverma]

@Levovar Thank You for detailed insights. Indeed helpful. You mentioned that you had considered this internally. What things did you consider and what blockers you guys faced, if any. Also, what were the reasons not to pursue this until now. Just trying to get as much information as possible, before we proceed.

[Levovar]

@mudverma well, besides that it always seems like we have higher priority items, I think we first wanted to make sure that if such a functionality is introduced, not everyone can influence, or change these settings once applied. So the new (ClusterNetwork, TenantNetwork) APIs were always considered higher prio, but that is done now.
The other was that besides traffic shaping, we also consider generic network policies as a valid use-case. Generic meaning again working with at least all dynamic backends.
As we always imagined network policing rules should be created inside the Pod's netns, traffic shaping and network policing becomes almost the same problem (i.e. provision certain iptables rules inside the Pod's netns); we were thinking about creating one definitive API to capture both requirements

but I think it is probably anyway better to separate these needs, one API for shaping, one for policing, with their own, independent, optional controllers.
so this is just a thought process we had in the past, but I see no real blockers today

if you do really implement such functionality, I think it would be best to have an online design discussion first once you have some ideas regarding APIs / architecture etc.
or a design proposal PR, or something similar
I think it is beneficial to discuss the framework before an implementation, tends to save some rework effort usually :)