Missing webserver-secret-key-secret

[zyst1988]

Official Helm Chart version

1.18.0 (latest released)

Apache Airflow version

2.10.5

Kubernetes Version

Latest

Helm Chart configuration

Hi,

In my setup I deploy workers to a separate name space to better control their resource provisioning and management.

In the new version of helm charts deploying worker pods alone doesn't seem to create one of the secrets which then stops the worker pods from starting. The missing secret is 'webserver-secret-key-secret'.

I tried manually adding it back in by overriding the values yaml but with no luck. Would it be possible to please check why this particular secret is not being created when deploying just workers?

Thanks
Mark

Docker Image customizations

I'm deploying workers to a separate name space.

What happened

No response

What you think should happen instead

No response

How to reproduce

Deploy just the worker pods

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[albundy83]

Hello, you should post all the parameters you used to deploy your helm chart

[prakass1]

What you could be using is to create webserverSecretKeySecretName pointing to a secret you create using kubernetes containing key as webserverSecretKey and value as the output from python3 -c 'import secrets; print(secrets.token_hex(16))'

You can create the secret in the namespaces for both worker and other services. Example below:

apiVersion: v1
kind: Secret
metadata:
  name:airflow-webserver-secret-key
type: Opaque
stringData:
  webserverSecretKey:output from `python3 -c 'import secrets; print(secrets.token_hex(16))'`

To be able to provide more specific answer, you might have to provide more information for one to replicate it

[zyst1988]

Hey thanks for all your replies. I tried setting the webserveer key through the webserverSecretKeySecretName as well as the in-built AIRFLOW__WEBSERVER__SECRET_KEY under enableBuiltInSecretEnvVars:. Neither of these settings appear to be used when only the worker pods are deployed.

The webserver secret seems to have some kind of dependency on one of the other services/pods - I would imagine it's most likely the webserver service itself, however I haven't tested it.

[albundy83]

Yes exactly, I think you have disabled webserver to achieve your setup with a namespace with only workers deployed.
But disabling webserver prevent webserver secret from being created.
Here the line
As @prakass1 write you, you must manually create the missing secret in your workers only namespace

[zyst1988]

Hard disagree:

1. Disabling the webserver did not cause any issues with secrets / global variables like the webserver key until version 3.0.
2. It makes no sense for global secrets / variables to stop working if one of the services get disabled. No other global secrets appear to have been affected like this.
3. Creating a separate key just for the worker is a terrible idea since the key needs to be shared with the webserver service.

This is clearly a bug and a global secret like the webserver key should not have been tied to webserver or any other single service which it was not until the version 3.0 where something happened to it.

Creating separate worker instances is a valid design pattern so this should be fully supported without having to resort to any workarounds.