-
Notifications
You must be signed in to change notification settings - Fork 16.4k
Description
Apache Airflow Helm Chart version
1.16.0
Apache Airflow version
2.10.5
Kubernetes Version
1.28+
Helm Chart configuration (Helm values)
workers:
persistence:
enabled: true
volumeClaimTemplates:
- metadata:
name: data
spec:
storageClassName: longhorn
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
extraVolumeMounts:
- name: data
mountPath: /dataDocker Image customizations
No response
What happened
It is impossible to add custom volumeClaimTemplates to workers (e.g., for a /data volume) due to the template logic.
The worker resource type and volumeClaimTemplates behavior depends on workers.persistence.enabled:
persistence.enabled |
Resource Type | Custom volumeClaimTemplates |
|---|---|---|
true (default) |
StatefulSet | Only renders default logs PVC, custom templates ignored |
false |
Deployment | Deployments do not support volumeClaimTemplates |
Result: There is no configuration that allows adding custom persistent volumes to workers.
What you think should happen instead
Users should be able to add custom volumeClaimTemplates (e.g., for /data) alongside the default logs volume when using a StatefulSet.
When persistence.enabled: true, the StatefulSet should append custom volumeClaimTemplates:
volumeClaimTemplates:
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: logs
spec:
...
{{- with .Values.workers.volumeClaimTemplates }}
{{- toYaml . | nindent 4 }}
{{- end }}How to reproduce
- Deploy the chart with the configuration above
- Pod fails with error:
volumeMounts[0].name: Not found: "data" - Setting
persistence.enabled: falsecreates a Deployment instead, which does not support volumeClaimTemplates at all
Root Cause
In templates/workers/worker-deployment.yaml, the volumeClaimTemplates block with custom templates is only rendered in the {{- else if not $persistence }} branch, but that branch creates a Deployment, not a StatefulSet.
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct