Worker always return Invalid auth token when setting up demo environment using docker compose

[muzy]

Apache Airflow version

3.0.0

If "Other Airflow 2 version" selected, which one?

No response

What happened?

I've installed Airflow 3.0.0 using the docker compose instructions as described here: https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html

I followed all steps and after running docker compose up, I logged into the admin user interface to run the example_simplest_dag . The DAG run fails and the worker shows this error log:

2025-04-23 18:14:50.194787 [info ] Task execute_workload[02ddde1a-8165-459e-91dd-518f2dd56c5d] received [celery.worker.strategy]
2025-04-23 18:14:50.304948 [info ] [02ddde1a-8165-459e-91dd-518f2dd56c5d] Executing workload in Celery: token='eyJ***' ti=TaskInstance(id=UUID('019663dc-130a-764a-ac1b-d956964ee946'), task_id='my_task', dag_id='example_simplest_dag', run_id='manual__2025-04-23T18:14:49.335659+00:00', try_number=1, map_index=-1, pool_slots=1, queue='default', priority_weight=1, executor_config=None, parent_context_carrier={}, context_carrier={}, queued_dttm=None) dag_rel_path=PurePosixPath('example_simplest_dag.py') bundle_info=BundleInfo(name='example_dags', version=None) log_path='dag_id=example_simplest_dag/run_id=manual__2025-04-23T18:14:49.335659+00:00/task_id=my_task/attempt=1.log' type='ExecuteTask' [airflow.providers.celery.executors.celery_executor_utils]
2025-04-23 18:14:50.694139 [error ] Task execute_workload[02ddde1a-8165-459e-91dd-518f2dd56c5d] raised unexpected: ServerResponseError('Invalid auth token: Signature verification failed') [celery.app.trace]

What you think should happen instead?

The DAG should return the result without errors.

How to reproduce

1. create a fresh virtual machine running ubuntu server and install docker
2. in an empty directory with a user allowed to run the dockercommands run the following steps:
3. run curl -LfO 'https://airflow.apache.org/docs/apache-airflow/3.0.0/docker-compose.yaml'
4. run mkdir -p ./dags ./logs ./plugins ./config
5. run echo -e "AIRFLOW_UID=$(id -u)" > .env
6. run `docker compose up airflow-init``
7. run docker compose up
8. if docker is not running on your machine, create ssh port forwarding to access the web-interface because otherwise you will run into this issue: Airflow 3.0.0 docker compose documentation does not show image version correctly #49622
9. login with default credentials, navigate to the example_simplest_dagDAG and trigger the run

Operating System

Ubuntu 24.04.2 LTS

Versions of Apache Airflow Providers

apache-airflow-providers-amazon | 9.6.1
apache-airflow-providers-celery | 3.10.6
apache-airflow-providers-cncf-kubernetes | 10.4.3
apache-airflow-providers-common-compat | 1.6.0
apache-airflow-providers-common-io | 1.5.4
apache-airflow-providers-common-messaging | 1.0.0
apache-airflow-providers-common-sql | 1.26.0
apache-airflow-providers-docker | 4.3.1
apache-airflow-providers-elasticsearch | 6.2.2
apache-airflow-providers-fab | 2.0.1
apache-airflow-providers-ftp | 3.12.3
apache-airflow-providers-git | 0.0.2
apache-airflow-providers-google | 15.1.0
apache-airflow-providers-grpc | 3.7.3
apache-airflow-providers-hashicorp | 4.1.1
apache-airflow-providers-http | 5.2.2
apache-airflow-providers-microsoft-azure | 12.3.1
apache-airflow-providers-mysql | 6.2.2
apache-airflow-providers-odbc | 4.9.2
apache-airflow-providers-openlineage | 2.2.0
apache-airflow-providers-postgres | 6.1.3
apache-airflow-providers-redis | 4.0.2
apache-airflow-providers-sendgrid | 4.0.1
apache-airflow-providers-sftp | 5.2.1
apache-airflow-providers-slack | 9.0.5
apache-airflow-providers-smtp | 2.0.3
apache-airflow-providers-snowflake | 6.2.2
apache-airflow-providers-ssh | 4.0.1
apache-airflow-providers-standard | 1.0.0

Deployment

Docker-Compose

Deployment details

docker compose version
Docker Compose version v2.35.1

docker version
Client: Docker Engine - Community
Version: 28.1.1
API version: 1.49
Go version: go1.23.8
Git commit: 4eba377
Built: Fri Apr 18 09:52:14 2025
OS/Arch: linux/amd64
Context: default

Server: Docker Engine - Community
Engine:
Version: 28.1.1
API version: 1.49 (minimum version 1.24)
Go version: go1.23.8
Git commit: 01f442b
Built: Fri Apr 18 09:52:14 2025
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.7.27
GitCommit: 05044ec0a9a75232cad458027ca83437aae3f4da
runc:
Version: 1.2.5
GitCommit: v1.2.5-0-g59923ef
docker-init:
Version: 0.19.0
GitCommit: de40ad0

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[prakass1]

I face the same issue using the docker-compose setup on mac as well.

[dheerajturaga]

For a workaround set export AIRFLOW__API_AUTH__JWT_SECRET=<some_random_string> you can generate one with
openssl rand -base64 16

@amoghrajesh, @jscheffl can you assign this to me, I can provide fix

[inkerinmaa]

set export AIRFLOW__API_AUTH__JWT_SECRET=<some_random_string>

Hi. I am setting this env variable in docker-compose but when I start containers all of them still have different jwt_secret in airflow.cfg

[alonahmias]

For a workaround set export AIRFLOW__API_AUTH__JWT_SECRET=<some_random_string> you can generate one with openssl rand -base64 16
it did not provide a fix for me

[dheerajturaga]

@inkerinmaa , @alonahmias
Something like this will work

@inkerinmaa , yes all the containers would have its own airflow.cfg with different jwt_secret ( thats what I'm trying to fix) but the env variable override should force all containers to pick the one set in the env

[ashb]

Might also be related to #49681

[alonahmias]

Might also be related to #49681

So yea i read that, what helped for me is removing all the volumes and leaving only the airflow home one