-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
volumeClaimGC OnWorkflowCompletion does not work when onExit/entrypoint template errors #10408
Comments
I managed to reproduce the issue and working on a fix |
Turns out it has nothing to do with workflow templates. Here is a workflow not using WorkflowTemplates which can reproduce the problem: apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: wft-pvc-
spec:
volumeClaimTemplates:
- metadata:
name: builddir
spec:
accessModes: [ "ReadWriteMany" ]
resources:
requests:
storage: 1Mi
volumeClaimGC:
strategy: OnWorkflowCompletion
entrypoint: whalesay
onExit: notify
templates:
- name: whalesay
container:
image: docker/whalesay:latest
command: [cowsay]
args: ["hello world"]
- name: notify
container:
image: docker/whalesay:latest
command: [sh, -c]
args: ["exit 0"]
volumeMounts:
- name: foo # <<< invalid since no volume `foo` @jiachengxu also pointed out that it could be affecting entrypoint as well. Need to test it. |
I just tested the entrypoint, and this affects it indeed. apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: wft-pvc-
spec:
volumeClaimTemplates:
- metadata:
name: builddir
spec:
accessModes: [ "ReadWriteMany" ]
resources:
requests:
storage: 1Mi
volumeClaimGC:
strategy: OnWorkflowCompletion
entrypoint: whalesay # <<< referencing the invalid template
onExit: notify
templates:
- name: whalesay
container:
image: docker/whalesay:latest
command: [cowsay]
args: ["hello world"]
volumeMounts:
- name: foo # <<< invalid since no volume `foo`
- name: notify
container:
image: docker/whalesay:latest
command: [sh, -c]
args: ["exit 0"] Then the workflow will be |
…ed. Fixes argoproj#10408 (argoproj#10424) Signed-off-by: Jiacheng Xu <xjcmaxwellcjx@gmail.com>
…ed. Fixes argoproj#10408 (argoproj#10424) Signed-off-by: Jiacheng Xu <xjcmaxwellcjx@gmail.com> Signed-off-by: goshado <goshatoo@gmail.com>
Pre-requisites
:latest
What happened/what you expected to happen?
Using
OnWorkflowCompletion
pvc cleanup strategy:The PVC will not be deleted if there is a template error for the
onExit
handler (and possibly other failure scenarios with onExit).The key parts about this issue seems to be a combination of:
When I convert the WorkflowTemplate to a Workflow, the PVC clean up happens expectedly.
Version
latest as of 1/27
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
Note that this seems to be a combination of WorkflowTemplates and onExit
Logs from the workflow controller
Logs from in your workflow's wait container
The text was updated successfully, but these errors were encountered: