Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate status.nodeInfo.kubeProxyVersion field #4004

Open
16 tasks done
HirazawaUi opened this issue May 15, 2023 · 33 comments
Open
16 tasks done

Deprecate status.nodeInfo.kubeProxyVersion field #4004

HirazawaUi opened this issue May 15, 2023 · 33 comments
Assignees
Labels
sig/network Categorizes an issue or PR as relevant to SIG Network. sig/node Categorizes an issue or PR as relevant to SIG Node. stage/beta Denotes an issue tracking an enhancement targeted for Beta status tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team
Milestone

Comments

@HirazawaUi
Copy link
Contributor

HirazawaUi commented May 15, 2023

Enhancement Description

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

@k8s-ci-robot k8s-ci-robot added the needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label May 15, 2023
@HirazawaUi
Copy link
Contributor Author

/sig node
/cc @danwinship
/assign

@k8s-ci-robot k8s-ci-robot added sig/node Categorizes an issue or PR as relevant to SIG Node. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels May 15, 2023
@HirazawaUi
Copy link
Contributor Author

/cc @thockin

@SergeyKanzhelev
Copy link
Member

/milestone v1.28

@k8s-ci-robot k8s-ci-robot added this to the v1.28 milestone May 26, 2023
@thockin thockin added the sig/network Categorizes an issue or PR as relevant to SIG Network. label Sep 27, 2023
@thockin
Copy link
Member

thockin commented Sep 27, 2023

What the staus on this? Alpha in 29?

@HirazawaUi
Copy link
Contributor Author

What the staus on this? Alpha in 29?

Yes, we'll be in Alpha at v1.29

@SergeyKanzhelev
Copy link
Member

/milestone v1.29
/label lead-opted-in

@k8s-ci-robot k8s-ci-robot modified the milestones: v1.28, v1.29 Sep 27, 2023
@k8s-ci-robot k8s-ci-robot added the lead-opted-in Denotes that an issue has been opted in to a release label Sep 27, 2023
@npolshakova
Copy link

Hello @HirazawaUi 👋, 1.29 Enhancements team here!

Just checking in as we approach enhancements freeze on 01:00 UTC, Friday, 6th October, 2023.

This enhancement is targeting for stage alpha for 1.29 (correct me, if otherwise)

Here's where this enhancement currently stands:

  • KEP readme using the latest template has been merged into the k/enhancements repo.
  • KEP status is marked as implementable for latest-milestone: 1.29. KEPs targeting stable will need to be marked as implemented after code PRs are merged and the feature gates are removed.
  • KEP readme has up-to-date graduation criteria
  • KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here).

The status of this enhancement is marked as tracked for enhancement freeze 🚀 . Please keep the issue description up-to-date with appropriate stages as well. Thank you!

@krol3
Copy link

krol3 commented Oct 2, 2023

Hi @HirazawaUi 👋, v1.29 Communication Release Team. I would like to check any plan about publication blogs related new features, removals, and deprecations for this release.

You need to open a PR placeholder in the website repository.
The deadline will be on Thursday 19th October 2023
Here the 1.29 Calendar

@HirazawaUi
Copy link
Contributor Author

Ok, I've opened a PR placeholder in the website repository, thanks for the tip!

@harshitasao
Copy link

Hi @HirazawaUi 👋, v1.29 Docs Shadow here
Does this enhancement work planned for v1.29 require any new docs or modification to existing docs?
If so, please follows the steps here to open a PR against dev-1.29 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday, 19 October 2023.
Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.
Thank you!

@HirazawaUi
Copy link
Contributor Author

I have opened a placeholder PR at k/website.

@AnaMMedina21
Copy link
Member

Hello @sanposhiho 👋, v1.29 Enhancements team here.

Just checking in as we approach code freeze at 01:00 UTC Wednesday 1st November 2023

Here's where this enhancement currently stands:

  • All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
  • All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

Missing Approval on kubernetes/website#43348 and kubernetes/kubernetes#120954

Also, please let me know if there are other PRs in k/k we should be tracking for this KEP.

With all this, the status of this KEP is At Risk for Code Freeze.

As always, we are here to help if any questions come up. Thanks!

@npolshakova
Copy link

Hello @sanposhiho👋, 1.29 Enhancements team here.

With all the implementation(code related) PRs merged as per the issue description, this enhancement is now marked as tracked for code freeze for the 1.29 Code Freeze! 🚀

The test freeze is 01:00 UTC Wednesday 15th November 2023 / 18:00 PDT Tuesday 14th November 2023. Please make sure all test PRs are merged in by then. Let me know if there are additional test PRs we should track. Thanks!

@kcmartin
Copy link

kcmartin commented Nov 14, 2023

@HirazawaUi @sanposhiho Hi, 1.29 Comms Team again! You indicated you were planning to write a blog post for this feature. A reminder that today (November 14, 2023) is the deadline for opening a Draft blog PR. We noticed Docs PRs, but no blog PR. Would you please update us about your plans?

@HirazawaUi
Copy link
Contributor Author

This is just a simple mini-KEP, I think there is no need for blog PR.

@salehsedghpour
Copy link
Contributor

/remove-label lead-opted-in

@k8s-ci-robot k8s-ci-robot added the stage/stable Denotes an issue tracking an enhancement targeted for Stable/GA status label Jun 4, 2024
@SergeyKanzhelev
Copy link
Member

/stage beta

@k8s-ci-robot k8s-ci-robot added stage/beta Denotes an issue tracking an enhancement targeted for Beta status lead-opted-in Denotes that an issue has been opted in to a release and removed stage/stable Denotes an issue tracking an enhancement targeted for Stable/GA status labels Jun 4, 2024
@sreeram-venkitesh
Copy link
Member

sreeram-venkitesh commented Jun 5, 2024

Hello @HirazawaUi 👋, v1.31 Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 14th June 2024 / 19:00 PDT Thursday 13th June 2024.

This enhancement is targeting for stage beta for v1.31 (correct me, if otherwise)

Here's where this enhancement currently stands:

  • KEP readme using the latest template has been merged into the k/enhancements repo.
  • KEP status is marked as implementable for latest-milestone: v1.31. KEPs targeting stable will need to be marked as implemented after code PRs are merged and the feature gates are removed.
  • KEP readme has up-to-date graduation criteria
  • KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here).

For this KEP, everything is taken care of in #4625. Please make sure that it is merged in time before the enhancements freeze.

The status of this enhancement is marked as at risk for enhancement freeze. I can mark it as tracked as soon as the above PR is merged however! Please link the relevant PRs for beta in the issue description as well.

If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you!

@sreeram-venkitesh
Copy link
Member

With #4625 merged, we can mark this KEP as tracked for enhancements freeze! 🎉

@mbianchidev
Copy link
Member

Hello again @HirazawaUi @danwinship (hope you both read this one after the other comments so that "again" makes sense 😄 )

👋 from the v1.31 Communications Team!
We'd love for you to opt in to write a feature blog about your enhancement!
Some reasons why you might want to write a blog for this feature include (but are not limited to) if this introduces breaking changes, is important to our users, or has been in progress for a long time and is graduating. Or like in this case a deprecation & removal.

To opt in, let us know and open a Feature Blog placeholder PR against the website repository by 3rd July, 2024. For more information about writing a blog see the blog contribution guidelines.

Note: In your placeholder PR, use XX characters for the blog date in the front matter and file name. We will work with you on updating the PR with the publication date once we have a final number of feature blogs for this release.

@sreeram-venkitesh sreeram-venkitesh added the tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team label Jun 24, 2024
@mbianchidev
Copy link
Member

mbianchidev commented Jun 25, 2024

Hello again @HirazawaUi @danwinship (hope you both read this one after the other comments so that "again" makes sense 😄 )

👋 from the v1.31 Communications Team! We'd love for you to opt in to write a feature blog about your enhancement! Some reasons why you might want to write a blog for this feature include (but are not limited to) if this introduces breaking changes, is important to our users, or has been in progress for a long time and is graduating. Or like in this case a deprecation & removal.

To opt in, let us know and open a Feature Blog placeholder PR against the website repository by 3rd July, 2024. For more information about writing a blog see the blog contribution guidelines.

Note: In your placeholder PR, use XX characters for the blog date in the front matter and file name. We will work with you on updating the PR with the publication date once we have a final number of feature blogs for this release.

Reminder of the 3rd of July deadline!
It's totally fine to also opt out if you don't think that writing a blog is useful for our users or if you don't have time (in that case team comms can also help you out 👀 )

@HirazawaUi
Copy link
Contributor Author

@mbianchidev Thanks for the heads up, but I see @a-mccarthy has already added a PR for the blog, I'll ask if my assistance is needed to complete it

@a-mccarthy
Copy link

@HirazawaUi just to clarify, we are planning to include this in the Deprecations and Removal blog for 1.31. However, if you feel that this change requires a separate, full blog article to help users understand the changes, please open placeholder PR. perfectly ok, if the full blog post is not needed. thanks! :)

@sreeram-venkitesh
Copy link
Member

Hey again @HirazawaUi 👋 v1.31 Enhancements team here,

Just checking in as we approach code freeze at 02:00 UTC Wednesday 24th July 2024 / 19:00 PDT Tuesday 23rd July 2024.

Here's where this enhancement currently stands:

  • All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
  • All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

List of associated PRs:

If you anticipate missing code freeze, you can file an exception request in advance.

This KEP is marked as Tracked for code freeze. Thanks a lot for keeping the issue description up to date! Also, please let me know if there are other PRs in k/k we should be tracking for this KEP. As always, we are here to help if any questions come up. Thanks!

@liggitt
Copy link
Member

liggitt commented Aug 15, 2024

I missed that this was wanting to promote to beta in 1.31 ... because this is clearing population of a stable API field, the minimum period from announcement (in 1.29) is 1 year, so 1.33 is the earliest this can default to clearing. I'll open a revert of the enablement for master / 1.31

@kannon92
Copy link
Contributor

I'm not sure with @liggitt point if this KEP will be worked on 1.32. Please let me know if you want reviews for the KEP from sig-node in 1.32.

@liggitt
Copy link
Member

liggitt commented Aug 28, 2024

I think it would stay as-is for 1.32 and default-enable in 1.33

ialidzhikov added a commit to ialidzhikov/gardener that referenced this issue Aug 30, 2024
…n` field

The `.status field.nodeInfo.kubeProxyVersion` is a lie since its initial introduction. The field is set by kubelet which cannot know the kube-proxy version or whether kube-proxy is running at all or not.
The `DisableNodeKubeProxyVersion` feature gate is enabled by default since K8s 1.31. The field is set to empty string in the Node status.
There is no added value in checking this field in the upgrade tests due to the reasons from above.

Ref kubernetes/enhancements#4004
ialidzhikov added a commit to ialidzhikov/gardener that referenced this issue Sep 5, 2024
…n` field

The `.status field.nodeInfo.kubeProxyVersion` is a lie since its initial introduction. The field is set by kubelet which cannot know the kube-proxy version or whether kube-proxy is running at all or not.
The `DisableNodeKubeProxyVersion` feature gate is enabled by default since K8s 1.31. The field is set to empty string in the Node status.
There is no added value in checking this field in the upgrade tests due to the reasons from above.

Ref kubernetes/enhancements#4004
ialidzhikov added a commit to ialidzhikov/gardener that referenced this issue Sep 5, 2024
…n` field

The `.status field.nodeInfo.kubeProxyVersion` is a lie since its initial introduction. The field is set by kubelet which cannot know the kube-proxy version or whether kube-proxy is running at all or not.
The `DisableNodeKubeProxyVersion` feature gate is enabled by default since K8s 1.31. The field is set to empty string in the Node status.
There is no added value in checking this field in the upgrade tests due to the reasons from above.

Ref kubernetes/enhancements#4004
ialidzhikov added a commit to ialidzhikov/gardener that referenced this issue Sep 16, 2024
…n` field

The `.status field.nodeInfo.kubeProxyVersion` is a lie since its initial introduction. The field is set by kubelet which cannot know the kube-proxy version or whether kube-proxy is running at all or not.
The `DisableNodeKubeProxyVersion` feature gate is enabled by default since K8s 1.31. The field is set to empty string in the Node status.
There is no added value in checking this field in the upgrade tests due to the reasons from above.

Ref kubernetes/enhancements#4004
@tjons
Copy link
Contributor

tjons commented Sep 16, 2024

Hi, enhancements lead here - I inadvertently added this to the 1.32 tracking board 😀. Please readd it if you wish to progress this enhancement in 1.32.

/remove-label lead-opted-in

@k8s-ci-robot k8s-ci-robot removed the lead-opted-in Denotes that an issue has been opted in to a release label Sep 16, 2024
@thockin thockin modified the milestones: v1.31, v1.33 Sep 16, 2024
ialidzhikov added a commit to ialidzhikov/gardener that referenced this issue Sep 25, 2024
…n` field

The `.status field.nodeInfo.kubeProxyVersion` is a lie since its initial introduction. The field is set by kubelet which cannot know the kube-proxy version or whether kube-proxy is running at all or not.
The `DisableNodeKubeProxyVersion` feature gate is enabled by default since K8s 1.31. The field is set to empty string in the Node status.
There is no added value in checking this field in the upgrade tests due to the reasons from above.

Ref kubernetes/enhancements#4004
gardener-prow bot pushed a commit to gardener/gardener that referenced this issue Oct 5, 2024
* Allow instantiating v1.31 Kubernetes clients

* Update `README.md` and `docs/usage/supported_k8s_versions.md` for K8s 1.31

* Maintain added feature gates

```
./hack/compare-k8s-feature-gates.sh 1.30 1.31

Feature gates added in 1.31 compared to 1.30:
AllowDNSOnlyNodeCSR
AllowInsecureKubeletCertificateSigningRequests
AnonymousAuthConfigurableEndpoints
AuthorizeNodeWithSelectors
AuthorizeWithSelectors
ConcurrentWatchObjectDecode
CoordinatedLeaderElection
DRAControlPlaneController
DisableAllocatorDualWrite
ImageVolume
ReloadKubeletServerCertificateFile
ResilientWatchCacheInitialization
ResourceHealthStatus
SupplementalGroupsPolicy
WatchCacheInitializationPostStartHook
```

* Maintain removed feature gates

```
./hack/compare-k8s-feature-gates.sh 1.30 1.31

Feature gates removed in 1.31 compared to 1.30:
APIPriorityAndFairness
CSIMigrationRBD
CSINodeExpandSecret
ConsistentHTTPGetHandlers
CustomResourceValidationExpressions
DefaultHostNetworkHostPortsInPodTemplates
InTreePluginAWSUnregister
InTreePluginAzureDiskUnregister
InTreePluginAzureFileUnregister
InTreePluginGCEUnregister
InTreePluginOpenStackUnregister
InTreePluginRBDUnregister
InTreePluginvSphereUnregister
JobReadyPods
ReadWriteOncePod
ServiceNodePortStaticSubrange
SkipReadOnlyValidationGCE
```

* Maintain locked to default feature gates

```
./hack/compare-k8s-feature-gates.sh 1.30 1.31

Feature gates locked to default in 1.31 compared to 1.30:
AppArmor                                 Default:  true
AppArmorFields                           Default:  true
DevicePluginCDIDevices                   Default:  true
DisableCloudProviders                    Default:  true
DisableKubeletCloudCredentialProviders   Default:  true
ElasticIndexedJob                        Default:  true
JobPodFailurePolicy                      Default:  true
KubeProxyDrainingTerminatingNodes        Default:  true
LogarithmicScaleDown                     Default:  true
PDBUnhealthyPodEvictionPolicy            Default:  true
PersistentVolumeLastPhaseTransitionTime  Default:  true
PodDisruptionConditions                  Default:  true
StatefulSetStartOrdinal                  Default:  true
```

* Maintain admission plugins

```
./hack/compare-k8s-admission-plugins.sh 1.30 1.31

Admission plugins added in 1.31 compared to 1.30:

Admission plugins removed in 1.31 compared to 1.30:
PersistentVolumeLabel
```

* Maintain API groups

```
./hack/compare-k8s-api-groups.sh 1.30 1.31

Kubernetes API group versions added in 1.31 compared to 1.30:
coordination.k8s.io/v1alpha1
resource.k8s.io/v1alpha3

Kubernetes API GVRs added in 1.31 compared to 1.30:
coordination.k8s.io/v1alpha1/leasecandidates
networking.k8s.io/v1beta1/ipaddresses
networking.k8s.io/v1beta1/servicecidrs
resource.k8s.io/v1alpha3/deviceclasses
resource.k8s.io/v1alpha3/podschedulingcontexts
resource.k8s.io/v1alpha3/resourceclaims
resource.k8s.io/v1alpha3/resourceclaimtemplates
resource.k8s.io/v1alpha3/resourceslices
storage.k8s.io/v1beta1/volumeattributesclasses

Kubernetes API group versions removed in 1.31 compared to 1.30:
resource.k8s.io/v1alpha2

Kubernetes API GVRs removed in 1.31 compared to 1.30:
resource.k8s.io/v1alpha2/podschedulingcontexts
resource.k8s.io/v1alpha2/resourceclaimparameters
resource.k8s.io/v1alpha2/resourceclaims
resource.k8s.io/v1alpha2/resourceclaimtemplates
resource.k8s.io/v1alpha2/resourceclasses
resource.k8s.io/v1alpha2/resourceclassparameters
resource.k8s.io/v1alpha2/resourceslices
```

* Maintain kube-controller-manager controllers

```
./hack/compute-k8s-controllers.sh 1.30 1.31

kube-controller-manager controllers added in 1.31 compared to 1.30:
Added Controllers for API Group [networking/v1beta1]: service-cidr-controller
Added Controllers for API Group [resource/v1alpha3]: resourceclaim-controller

kube-controller-manager controllers removed in 1.31 compared to 1.30:
Removed Controllers for API Group [networking/v1alpha1]: service-cidr-controller
Removed Controllers for API Group [resource/v1alpha2]: resourceclaim-controller
```

* [no-op] Maintain copies of the DaemonSet controller's scheduling logic

* Add K8s 1.31 to the local CloudProfile

* tests: Don't check for Node's `.status field.nodeInfo.kubeProxyVersion` field

The `.status field.nodeInfo.kubeProxyVersion` is a lie since its initial introduction. The field is set by kubelet which cannot know the kube-proxy version or whether kube-proxy is running at all or not.
The `DisableNodeKubeProxyVersion` feature gate is enabled by default since K8s 1.31. The field is set to empty string in the Node status.
There is no added value in checking this field in the upgrade tests due to the reasons from above.

Ref kubernetes/enhancements#4004

* Add version constraints for K8s 1.31

* maintenance controller: Set `.spec.kubernetes.kubeAPIServer.oidcConfig.clientAuthentication=nil` when doing forceful update to K8s 1.31+

* maintenance controller: Move `kubernetes.kubelet.systemReserved` to `kubernetes.kubelet.kubeReserved` when doing forceful update to K8s 1.31+

* Nit: Use the KubeProxyEnabled helper func instead of duplicating the same logic

* Update the e2e tests section in `docs/development/new-kubernetes-version.md`

* Update provider extensions instructions in `docs/development/new-kubernetes-version.md`

* Remove unnecessary logic for the `KubeletCgroupDriverFromCRI` feature gate

Only the feature gate enablement is not enough the new auto-detection flow to be used. The feature depends on a new CRI API that will be present only in containerd 2.0+. Additionally, even if cgroup driver is specified and the new flow auto-detects another one from the CRI, it will ignore the specified cgroup driver and will use the auto-detected one.

* Default kubelet's and containerd's cgroup driver to `systemd` for K8s 1.31+

* Update the local Garden Kubernetes version to 1.31

* Nit: Do not log admission is being deployed on make gardener-extensions-down

* Order API groups alphabetically
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/network Categorizes an issue or PR as relevant to SIG Network. sig/node Categorizes an issue or PR as relevant to SIG Node. stage/beta Denotes an issue tracking an enhancement targeted for Beta status tracked/yes Denotes an enhancement issue is actively being tracked by the Release Team
Projects
Status: Tracked for Code Freeze
Status: Alpha
Status: Not for release
Status: Tracked for Doc Freeze
Development

No branches or pull requests