Airflow 3 current: Referring to #50385 solution, GitDagBundle does not inherits installed packages from the virtual enviroinment on which Airflow is running

[gbonazzoli]

Apache Airflow version

3.0.1

If "Other Airflow 2 version" selected, which one?

No response

What happened?

Please refer to #50385 solution,

@uranusjr

I've applied the patch to my 3.0.1 installation (processor.py and settings.py) and I think that the problem in not completely solved.

After the patch the module is found, but now if the module's code on the git repository has "import requirements", I think that those requirements are not inherited from the virtual enviroment on which Airflow is running.

This is the new error:

But the pyvmomi package is already installed has you can see:

(venv) root@airflow-tre:~# pip install pyvmomi
Requirement already satisfied: pyvmomi in ./venv/lib/python3.12/site-packages (8.0.3.0.1)
Requirement already satisfied: six>=1.7.3 in ./venv/lib/python3.12/site-packages (from pyvmomi) (1.17.0)
(venv) root@airflow-tre:~# python3
Python 3.12.3 (main, Feb  4 2025, 14:48:35) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyVim.connect import Connect, Disconnect
>>> 
(venv) root@airflow-tre:~# 

What you think should happen instead?

GitDagBundle has to supply DAGS in the same way and in the same python environment as a normal "dags" filesystem directory.

This statement must be valid also for a Virtual environment in which Airflow is running.

How to reproduce

Please look at #50385

Operating System

Ubuntu 24.04.2 LTS

Versions of Apache Airflow Providers

apache-airflow==3.0.1
apache-airflow-core==3.0.1
apache-airflow-providers-celery==3.10.6
apache-airflow-providers-common-compat==1.6.1
apache-airflow-providers-common-io==1.5.4
apache-airflow-providers-common-sql==1.27.0
apache-airflow-providers-git==0.0.2
apache-airflow-providers-postgres==6.1.3
apache-airflow-providers-redis==4.0.2
apache-airflow-providers-smtp==2.0.3
apache-airflow-providers-ssh==4.0.1
apache-airflow-providers-standard==1.1.0
apache-airflow-task-sdk==1.0.1

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[uranusjr]

Hmm, I can’t reproduce. Looks like the site-packages directory is correctly added to the task runner, from I can tell.

Can you run subprocess.run([sys.executable, "-m", "pip", "list"]) in a task and show the output for me?