-
Notifications
You must be signed in to change notification settings - Fork 8.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Job resources are not automatically cleaned up preventing version upgrades #7118
Comments
I see that my suggestion of using ingress-nginx/charts/ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml Lines 13 to 16 in 3dd7c83
However, this does not end up in the generated static manifests: ingress-nginx/deploy/static/provider/cloud/deploy.yaml Lines 623 to 626 in 3dd7c83
In order for this to work we'd need to start setting
|
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten |
Not stale /remove-lifecycle rotten |
Can we get some eyes on this? |
/lifecycle active @NeckBeardPrince see this comment from @rikatz: #7128 (comment) |
As a temporary workaround, I added the following steps after I deploy the controller: kubectl patch -n ingress-nginx job ingress-nginx-admission-create -p '{"spec":{"ttlSecondsAfterFinished":0}}'
kubectl patch -n ingress-nginx job ingress-nginx-admission-patch -p '{"spec":{"ttlSecondsAfterFinished":0}}' If you are using PowerShell on Windows, don't forget to escape the quotes differently: kubectl patch -n ingress-nginx job ingress-nginx-admission-create -p '{\"spec\":{\"ttlSecondsAfterFinished\":0}}'
kubectl patch -n ingress-nginx job ingress-nginx-admission-patch -p '{\"spec\":{\"ttlSecondsAfterFinished\":0}}' |
/triage accepted |
@iamNoah1: GuidelinesPlease ensure that the issue body includes answers to the following questions:
For more details on the requirements of such an issue, please see here and ensure that they are met. If this request no longer meets these requirements, the label can be removed In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
In our config we are kustomize. For this component we are applying a patch, e.g.: patches:
- target:
kind: Job
patch: |-
- op: add
path: /spec/ttlSecondsAfterFinished
value: 0 |
Looks like newer versions of k8s do not ship the batch/v1alpha1 version anymore and therefore the capacilities flag does not work anymore. The following patch should do the trick:
|
@robert-heinzmann-logmein wanna send the PR? Then just tag me and I can approve it. Also we dont support k8s <v1.20 anymore so fully dropping this API should be good |
Not sure, how about #7128 ? Following the discussions, I think as long as you support 1.20 (where this is still alpha) the toggle can not be removed right ? |
Oh it is beta only on v1.21? :/ yeah we should wait until v1.25 and drop support to v1.20 |
Also apparently that PR is stale, so if you are willing to open a new one lmk. |
It seems 1.25 is out, so this looks unblocked. |
This issue has not been updated in over 1 year, and should be re-triaged. You can:
For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/ /remove-triage accepted |
Enough number of upgrades are safely assumed to have been performed since the last activity on this issue and the version of the controller + the version of the Kubernetes referred to here, is no longer supported or too old to support. Since nobody is reporting upgrade failures due to pending jobs, I will close the issue for now. If a user posts data proving that upgrades fail due to pending jobs, on a kind cluster or a minikube cluster, then we can reopen this issue. /close |
@longwuyuan: Closing this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
NGINX Ingress controller version:
v0.45.0
Kubernetes version (use
kubectl version
):v1.19.0
Environment:
kustomize
andkpt live apply
What happened:
First deployment of
ingress-nginx
from static manifests is successful. When attempting to upgrade to any newer version of the static manifests (in our casev0.44.0 -> v0.45.0
) you will see errors indicating that job specs are immutable:I think this is simply because labels like
app.kubernetes.io/version
andhelm.sh/chart
are changing between versions.We are using
kpt pkg
commands to pull the manifests maintained indeploy/static/provider/cloud
into our own git repo, we apply kustomizations on top of that, and deploy withkpt live
commands.What you expected to happen:
I believe helm users do not experience this problem because of the
helm.sh/hook-delete-policy
annotations specified on the variousJob
resources. These is obviously not respected when deploying static manifests using other tooling.I would expect users deploying from static manifests to have a similar experience when upgrading between versions.
Rather than relying on proprietary helm hooks to ensure cleanup of the
Job
resources, perhaps we can usespec.ttlSecondsAfterFinished
in order to ensure these resources are pruned in a portable fashion.How to reproduce it:
This is not precisely what we are doing, but it should be reproducible like so:
/kind bug
The text was updated successfully, but these errors were encountered: