Skip to content

[helm-chart] 1.16.0: Cannot add custom volumeClaimTemplates to workers #59571

@m4dm4rtig4n

Description

@m4dm4rtig4n

Apache Airflow Helm Chart version

1.16.0

Apache Airflow version

2.10.5

Kubernetes Version

1.28+

Helm Chart configuration (Helm values)

workers:
  persistence:
    enabled: true
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        storageClassName: longhorn
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 10Gi
  extraVolumeMounts:
    - name: data
      mountPath: /data

Docker Image customizations

No response

What happened

It is impossible to add custom volumeClaimTemplates to workers (e.g., for a /data volume) due to the template logic.

The worker resource type and volumeClaimTemplates behavior depends on workers.persistence.enabled:

persistence.enabled Resource Type Custom volumeClaimTemplates
true (default) StatefulSet Only renders default logs PVC, custom templates ignored
false Deployment Deployments do not support volumeClaimTemplates

Result: There is no configuration that allows adding custom persistent volumes to workers.

What you think should happen instead

Users should be able to add custom volumeClaimTemplates (e.g., for /data) alongside the default logs volume when using a StatefulSet.

When persistence.enabled: true, the StatefulSet should append custom volumeClaimTemplates:

volumeClaimTemplates:
  - apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: logs
    spec:
      ...
  {{- with .Values.workers.volumeClaimTemplates }}
    {{- toYaml . | nindent 4 }}
  {{- end }}

How to reproduce

  1. Deploy the chart with the configuration above
  2. Pod fails with error: volumeMounts[0].name: Not found: "data"
  3. Setting persistence.enabled: false creates a Deployment instead, which does not support volumeClaimTemplates at all

Root Cause

In templates/workers/worker-deployment.yaml, the volumeClaimTemplates block with custom templates is only rendered in the {{- else if not $persistence }} branch, but that branch creates a Deployment, not a StatefulSet.

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions