Rewrite "The Kubernetes network model" #47903

danwinship · 2024-09-12T21:01:03Z

Spun off from #41419 which in turn is spun off from #39890. Whee!

This just focuses on improving the existing text, and doesn't (much) add anything to it (leaving that for the two older PRs).

The big problem with this page is that it is very very very old, and that shows in two main ways:

It is super focused on explaining Kubernetes networking to people who are used to 2015-era single-host docker networking, but it never actually explains that that's what it's contrasting against. I'm sure that when this was first written, it was incredibly helpful, but at this point, it is just confusing to the vast majority of readers.
Especially toward the end of the doc, the hyperlinks are very weird. It looks like this is the result of gradual refactoring of the docs; someone linked to document X for its discussion of topic Y, but then later the discussion of topic Y was moved into a different document, leaving the old link now pointing to the wrong docs, etc.

I tried to keep most of the existing information, but some of the "kubernetes-is-not-docker" type stuff just didn't really seem to fit well and got dropped. (But if anyone thinks I dropped something important, let me know and I'll try to find a place for it.)

The update completely reorganizes the existing information, essentially moving the old "Kubernetes networking addresses four concerns" list to the top and putting the "fundamental requirements on any networking implementation" list inside it. And I expanded heavily on the "requirements on any networking implementation" / "detail of the particular container runtime in use" comments to talk about what parts of the network model are core to k8s and what parts are implemented externally.

I'm sure there are style issues to be fixed... (It may need some better transitions in places, or subheadings?)

/cc @sftim @aojea @thockin

netlify · 2024-09-12T21:10:27Z

✅ Pull request preview available for checking

Built without sensitive environment variables

Name	Link
🔨 Latest commit	`5c9bb88`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-io-main-staging/deploys/66e8ad8e555de8000845369b
😎 Deploy Preview	https://deploy-preview-47903--kubernetes-io-main-staging.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

sftim

Thanks. We've updated the style guide since this page was last updated; please follow the current style guide around how we write API kinds (and also, ideally, avoid Latin abbreviations).

/lgtm

This could merge as-is; it's an obvious improvement on what we have now.

content/en/docs/concepts/services-networking/_index.md

sftim · 2024-09-13T12:22:02Z

content/en/docs/concepts/services-networking/_index.md

+      * All pods can communicate with all other pods, whether on the
+        same [node](/docs/concepts/architecture/nodes/) or on
+        different nodes, without NAT.
+
+          * (For platforms that support host-network pods (e.g.
+            Linux), this rule applies to host-network pods as well.)


Aligning to the style guide:

Suggested change

* All pods can communicate with all other pods, whether on the

same [node](/docs/concepts/architecture/nodes/) or on

different nodes, without NAT.

* (For platforms that support host-network pods (e.g.

Linux), this rule applies to host-network pods as well.)

* All pods have a route available to all other pods, whether two Pods are on the

same [node](/docs/concepts/architecture/nodes/) or on

different nodes. This is a direct means to communicate (no NAT).

* (For platforms that support host-network pods (Linux),

this rule applies to host-network pods as well.)

Pods may or may not be able to actually communicate, due to:

packet filtering (and similar controls)

misconfiguration, I suppose

outages

I brought back the "barring intentional network segmentation" text from before. That applies to the "agents on a node" subclause too, so I put it at the "the pod network handles communication" level rather than the "all pods can communicate" level.

On the Linux thing, it felt weird to be calling out the Linux behavior, which is the expected behavior, rather than the Windows behavior, which is considered wrong, so I flipped that.

content/en/docs/concepts/services-networking/_index.md

k8s-ci-robot · 2024-09-13T12:30:12Z

LGTM label has been added.

Git tree hash: 063c93cc8e27413af8eabfba02c66827d79d4516

content/en/docs/concepts/services-networking/_index.md

aojea · 2024-09-13T15:03:24Z

content/en/docs/concepts/services-networking/_index.md

@@ -80,7 +81,7 @@ corresponding functionality is provided by external components:
    On Linux, most container runtimes use the
    {{< glossary_tooltip text="Container Networking Interface (CNI)" term_id="cni" >}}
    to interact with the pod network implementation, so these
-    implementations are often called "CNI plugins"
+    implementations are often called _CNI plugins_.


I'm surprised you didn't added "network plugins" here too :)

aojea · 2024-09-13T15:04:36Z

LGTM module one comment about the meaning of "long lived" in services, but because I don't understand well the meaning of that expression there

sftim · 2024-09-13T15:58:14Z

I judge that even where there's room to improve further, this makes the docs better than they are as they stand.

/lgtm

k8s-ci-robot · 2024-09-13T15:58:21Z

LGTM label has been added.

Git tree hash: 4e404a89c91bf4f6f0fb44679383ac6c004c9fce

danwinship · 2024-09-13T17:20:35Z

/hold
(still planning to address the remaining comments, I just didn't get to that yet)

salaxander · 2024-09-13T19:07:49Z

Thanks for all the work on this @danwinship!

salaxander · 2024-09-13T19:08:38Z

For next week's PR wrangler - I think this looks great so far, when Dan is happy with the state of it, I think it's good to merge

danwinship · 2024-09-14T16:48:19Z

/hold cancel

aojea · 2024-09-14T18:03:31Z

content/en/docs/concepts/services-networking/_index.md

+      * A service proxy implementation, takes care of converting the
+        `Service` and `EndpointSlice` objects into packet-rewriting
+        rules using appropriate operating system APIs.


There can be service proxy implemenations that are userspace can we rephrase this to be less low level "A service proxy implementation uses the Services and EndpointSlices object to program the dataplane using appropriate operating system APIs" or something like that

This is just an overview, and given that all official proxy implementations and commonly-used third-party proxy implementations rewrite packets, it seems reasonable to say it this way.

I agree with @danwinship, and also the data plane programming might be done by something even more custom (for example: cloud provider APIs; compiling P4 code and reconfiguring external hardware).

fair enough

aojea · 2024-09-14T18:05:24Z

content/en/docs/concepts/services-networking/_index.md

+corresponding functionality is provided by external components, some
+of which are optional:
+
+  * Pod network namespace setup is handled by system-level software


Suggested change

* Pod network namespace setup is handled by system-level software

* Pod network namespace setup is handled by container runtimes that use the [Container Runtime

Interface] to communicate with the kubelet

sftim

If we merge this as-is, the docs become better.

/lgtm

content/en/docs/concepts/services-networking/_index.md

k8s-ci-robot · 2024-09-15T09:54:59Z

LGTM label has been added.

Git tree hash: 6f2ebf5fe09a73c9955a04f00883c3e566ea7dde

sftim · 2024-09-15T20:23:57Z

(still)
/lgtm

k8s-ci-robot · 2024-09-15T20:24:02Z

LGTM label has been added.

Git tree hash: 60497eeb49cd40ae2fc858096ef86fcd713d83b1

thockin

Thanks, this was WAY overdue

content/en/docs/concepts/services-networking/_index.md

thockin · 2024-09-16T21:11:48Z

content/en/docs/concepts/services-networking/_index.md

+    [Ingress](/docs/concepts/services-networking/ingress/)) allows
+    you to make Services accessible to clients that are outside the cluster.
+
+      * A simpler, but less-configurable, mechanism for cluster


Are we comfortable implying Service LB is obsolete? Does L4 Gateway handle EVERYTHING that LB does? I don't think so...

What change would you recommend?

I definitely wasn't trying to imply that it's obsolete (and note that the old version of this page doesn't link to the Service LB docs at all, so this is actually making that feature more prominent).

But if we're going to have a single top-level answer for "how cluster ingress works in Kubernetes", it should be Gateway, right?

(I could demote the Ingress reference to a nested bullet point... or remove it.)

if we're going to have a single top-level answer for "how cluster ingress works in Kubernetes", it should be Gateway, right?

I can't see how that is true until L4 gateway is as complete as service LB? @robscott

As for wording here, something like:

"An alternative mechanism for exposing a single Service is the Service API's type: LoadBalancer. It is simpler than Gateway, but less powerful, and is missing some exact functional equivalents."

Or something? I know we don't want to make this "at the time of this writing" but the reality is that, at the time of this writing Gateway does not fully replace type: LB.

I think this PR makes a better compromise than the page as it stands (and obviously so), so I strongly support merging as-is and then iterating.

I don't see anything that is so wrong that it would seriously mislead readers or get in the way of their learning.

Co-authored-by: Tim Bannister <tim@scalefactory.com>

danwinship · 2024-09-16T22:13:50Z

content/en/docs/concepts/services-networking/_index.md

+      * A service proxy implementation monitors the `Service` and
+        `EndpointSlice` objects, and programs the data plane to route
+        service traffic to its backends, by using operating system or
+        cloud provider APIs to intercept or rewrite packets.


how's that?

reylejano · 2024-09-18T05:52:07Z

/label refactor
/sig network
This PR is a wonderful improvement. The Kubernetes network model is easier to understand with this PR (imo).

This PR has reviews from SIG Network co-chair and tech lead.
/approve

k8s-ci-robot · 2024-09-18T05:52:19Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: reylejano

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~content/en/docs/OWNERS~~ [reylejano]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

aojea · 2024-09-18T06:50:23Z

LGTM, @thockin for the final approval

sftim · 2024-09-18T09:09:34Z

I can't imagine @thockin objecting at this point, so:

/lgtm

k8s-ci-robot · 2024-09-18T09:09:40Z

LGTM label has been added.

Git tree hash: 260f755aa2712c1cab64d7cc2e2bb320a8d0c680

thockin · 2024-09-18T17:47:46Z

LGTM

k8s-ci-robot requested review from aojea, sftim and thockin September 12, 2024 21:01

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. language/en Issues or PRs related to English language size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Sep 12, 2024

danwinship mentioned this pull request Sep 12, 2024

Explain Kubernetes network model in networking concept index #41419

Draft

sftim reviewed Sep 13, 2024

View reviewed changes

k8s-ci-robot assigned sftim Sep 13, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 13, 2024

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 13, 2024

k8s-ci-robot requested a review from sftim September 13, 2024 13:43

aojea reviewed Sep 13, 2024

View reviewed changes

content/en/docs/concepts/services-networking/_index.md Show resolved Hide resolved

aojea reviewed Sep 13, 2024

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 13, 2024

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 13, 2024

danwinship force-pushed the rewrite-network-model branch from 3805035 to ad46267 Compare September 14, 2024 16:46

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 14, 2024

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 14, 2024

aojea reviewed Sep 14, 2024

View reviewed changes

sftim reviewed Sep 15, 2024

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 15, 2024

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 15, 2024

k8s-ci-robot requested a review from sftim September 15, 2024 18:48

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 15, 2024

thockin reviewed Sep 16, 2024

View reviewed changes

Rewrite "The Kubernetes network model"

5c9bb88

Co-authored-by: Tim Bannister <tim@scalefactory.com>

danwinship force-pushed the rewrite-network-model branch from e849491 to 5c9bb88 Compare September 16, 2024 22:13

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 16, 2024

danwinship commented Sep 16, 2024

View reviewed changes

k8s-ci-robot added sig/network Categorizes an issue or PR as relevant to SIG Network. refactor Indicates a PR with large refactoring changes e.g. removes files or moves content labels Sep 18, 2024

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 18, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 18, 2024

k8s-ci-robot merged commit 719e87a into kubernetes:main Sep 18, 2024
6 checks passed

danwinship deleted the rewrite-network-model branch September 18, 2024 09:47

asem-hamid mentioned this pull request Sep 18, 2024

Sub-Umbrella Issue: List of Prioritized Content to Localize in Bangla #47747

Open

	* Pod network namespace setup is handled by system-level software
	* Pod network namespace setup is handled by container runtimes that use the [Container Runtime
	Interface] to communicate with the kubelet

Rewrite "The Kubernetes network model" #47903

Rewrite "The Kubernetes network model" #47903

Conversation

danwinship commented Sep 12, 2024 • edited Loading

netlify bot commented Sep 12, 2024 • edited Loading

✅ Pull request preview available for checking

sftim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

k8s-ci-robot commented Sep 13, 2024

Choose a reason for hiding this comment

aojea commented Sep 13, 2024

sftim commented Sep 13, 2024

k8s-ci-robot commented Sep 13, 2024

danwinship commented Sep 13, 2024

salaxander commented Sep 13, 2024

salaxander commented Sep 13, 2024

danwinship commented Sep 14, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sftim left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Sep 15, 2024

sftim commented Sep 15, 2024

k8s-ci-robot commented Sep 15, 2024

thockin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thockin Sep 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

reylejano commented Sep 18, 2024

k8s-ci-robot commented Sep 18, 2024

aojea commented Sep 18, 2024

sftim commented Sep 18, 2024

k8s-ci-robot commented Sep 18, 2024

thockin commented Sep 18, 2024

danwinship commented Sep 12, 2024 •

edited

Loading

netlify bot commented Sep 12, 2024 •

edited

Loading

thockin Sep 17, 2024 •

edited

Loading