Add diagrams to The Kubernetes Network Model

[chrismetz09]

This is a Feature Request

What would you like to be added
Add diagrams to the The Kubernetes network model paragraphs.

Why is this needed
Readers, new and experienced, may find it difficult to visualize the concepts described in the Kubernetes network model intro discussion.

A couple of simple diagrams will assist readers in understanding and preparing them for the subsequent explanations of the Kubernetes networking concerns referenced at the bottom of the page.

Comments
Here is an example-hack diagram to accompany the Kubernetes imposes the following ... paragraphs. A diagram referral and caption described in the How to use captions section of the Diagram guide will further help the reader grasp this concept.

Mermaid live editor link

---

Here is an example-hack diagram to accompany the IP-per-pod model description. Again, a diagram referral and caption will help the reader grasp this concept.

Mermaid live editor link

[mk46]

/sig network

[sftim]

We could take inspiration - without any plagiarism, of course - from https://blog.neuvector.com/article/advanced-kubernetes-networking.

/triage accepted
/language en
/priority important-longterm

[sftim]

It'd be nice to mix in some IPv6 examples. Kubernetes lets you use IPv6-only, IPv4-only or even dual-stack networking.

[PriyanshuAhlawat]

/assign

[chrismetz09]

It'd be nice to mix in some IPv6 examples. Kubernetes lets you use IPv6-only, IPv4-only or even dual-stack networking.

@sftim, I view this page as an intro-high-level intro that tees up the 4 x requirements discussion and more network concepts. IMHO, particular with k8s-cloud-network-newbies, keeping it simple would be appropriate for start.

Diagrams on IPv6, IPv4, dual stack best belong (and badly needed) in /docs/concepts/services-networking/dual-stack/.

[sftim]

@chrismetz09 I think you're recommending that we treat IPv4 as the stack we prefer and document, and relegate discussion of IPv6 to separate pages.

We try not to favor one technology over another, overall. With that in mind, I'd like to see at least some mention of IPv6 being an option in the overview page. It might be that we can't find a way to get it in without confusing readers; however, our starting point should be to try. If illustrating things ends up easier where we've used all-IPv4 for an example, that's OK. In that case, let's add some text to explain that detail.

I'm not sure if we can use images within tabs, but if we can, then we may actually be able to illustrate all three options and let readers compare.

[sftim]

One key thing we should aim to convey with diagrams is that every pod in a cluster can address every other pod in a cluster; packets from one pod to any other pod should never yield a “no route to host” error.

[chrismetz09]

@sftim, thinking that classic container eth0, cbr0, vethx, etc. diagram is waaaay too complex for initial K8s network model discussion on this page

Could lead complexity bloat quickly. This should be placed in new separate section, maybe combined with CNI, etc.

[chrismetz09]

One key thing we should aim to convey with diagrams is that every pod in a cluster can address every other pod in a cluster; packets from one pod to any other pod should never yield a “no route to host” error.

Totally.

[chrismetz09]

@PriyanshuAhlawat, thanks for taking this on. If possible, please use Mermaid to hack the diagrams. This will facilitate collaboration and reviews for this interesting and important topic. Tx!

[sftim]

Work on this important issue is very welcome. If anyone's made a start, please share what you have - even if it's not finished, that could help someone else move the work forward.

[chetak123]

Hey @PriyanshuAhlawat Are you still working on this issue, I would like to collaborate on this issue please.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[chrismetz09]

/assign

[chrismetz09]

/remove-lifecycle stale
/assign

[chrismetz09]

Looking over https://kubernetes.io/docs/concepts/services-networking/.

New and/or experienced network person looking over K8s networking for the first time could find this difficult to parse.

Simple pictures plus text edits is necessary. More detailed pictures with veths, namespaces, cbr0, cni etc. can be added at the bottom. This tees up the cluster networking discussion.

I will hack up some pictures to illustrate.

[sftim]

@chrismetz09 you might like to look at https://deploy-preview-30817--kubernetes-io-main-staging.netlify.app/docs/concepts/services-networking/service/ which illustrates how I hope the page will look within 12 months or so (see PR #30817)

The existing diagrams have moved into the reference section: https://deploy-preview-30817--kubernetes-io-main-staging.netlify.app/docs/reference/networking/virtual-ips/
I was hoping to follow this up with more work to redo those diagrams, once the constituent elements of #30817 merge into main.

[sftim]

I don't have plans to make any significant change to https://kubernetes.io/docs/concepts/services-networking/

[sftim]

I do recommend putting more advanced / detailed information inside https://k8s.io/docs/reference/networking/

[chrismetz09]

I do recommend putting more advanced / detailed information inside https://k8s.io/docs/reference/networking/

Got it, will go with that approach.

[chrismetz09]

#36671 generates 404.

[chrismetz09]

Some initial pointers:

veth
k8s docs namespaces
linux network namespace
k8s docs cluster networking
k8s docs pod networking

[sftim]

We might want to distinguish between:

• essential to the network model (eg: a primary network interface; IP networking)
• typical to implementations of the Kubernetes network model (eg: using a veth pair; bridging; 192.168.0.0/16)

[chrismetz09]

Exactly what I was thinking! Keeping it simple we have:

K8s network model: "IP-per-Pod", localhost for containers in same pod namespace, L2/IP networking for Inter-pod.
K8s network model implementations. Here is a hack figure for single host communications. More text and figures forthcoming.

figure link in diagrams.net

[sftim]

• It'd be nice to illustrate an example using IP-only networking, eg a tun and some imaginary userland packet mangler outside the Pod netns.
• The host interface need not be in the root netns; if we can avoid people wrongly inferring that it must be, even better.

[chrismetz09]

Template diagram for pod networking between different hosts. This can be broken up into multiple diagrams, perhaps showing one with CNI-specific encaps and one with native L2/IP networking. Again just a template. Looking to keep it simple for readers wishing to understand the k8s network model.

The diagrams and text would be added to k8s network model page.

figure link in diagrams.net

[sftim]

Feedback at this point is maybe don't use names veth1 and cbr0 in the host ns. How about pod1-host (or pod4-host) and l2bridge0? Also, in the writeup, explain that the two bridge interfaces are unrelated.

[sftim]

@chrismetz09 if you're happy to, please feel free to open a PR.

[mehabhalodiya]

@PriyanshuAhlawat I don't see any updates; so unassigning you. Please feel free to assign, if you come back here again and are willing to work on 🙂
/unassign @PriyanshuAhlawat

[k8s-triage-robot]

This issue has not been updated in over 1 year, and should be re-triaged.

You can:

• Confirm that this issue is still relevant with /triage accepted (org members only)
• Close this issue with /close

For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/

/remove-triage accepted

[a-mccarthy]

/triage accepted

@sftim looks like you were the last one working on this in #41419, can you comment here on what you feel is still needed for this issue?

[sftim]

It's tricky because this isn't about documenting a thing people already agree on. Instead, we need to find that agreement partly by proposing a model and then seeing what concerns people have.

What we need to find is a shared understanding of what is core to Kubernetes networking. We need that understanding, perhaps as a sketch or wireframe or as a writeup, so that we can explain it and agree on how to explain it.

This isn't a small ask, IMO. I can probably make time to facilitate a discussion around this, or to help smooth out disagreements on where the boundaries are. Maybe some other folks can too.

Specifically, we need to take things out of the model. We won't show layer 2 (Kubernetes doesn't have opinions here), or packet filtering (Kubernetes doesn't really require it), or iptables. We really would prefer to abstract away kube-proxy because that is an implementation rather than the underlying model. It needs effort and it needs time from the key stakeholders (Docs, Network, Architecture).