PythonVirtualenvOperator Warning Failed to Delete venv

[RaphCodec]

Apache Airflow version

3.1.0

If "Other Airflow 2 version" selected, which one?

No response

What happened?

I am running Airflow 3.1.0 with the LocalExecutor, using Docker Compose. I have a simple DAG shown below just to test out the PythonVirtualenvOperator. When I run the DAG Airflow automatically uses UV to create the venv in a tmp directory. The DAG then runs sucessfully, but at the end there is a warning that the venv failed to be deleted becuase Airflow for some reason is now looking at a different directory for the venv.

Originally the venv was create in /tmp but the airflow tried to delete the venv from /opt/airflow/tmp. Therefore I tried setting the UV_Cache_DIR to /opt/airflow/tmp but now Airflow tries to delete the venv from /opt/airflow/opt/airflow/tmp.

Log with venv delete failure:

INFO - DAG bundles loaded: dags-folder
INFO - Filling up the DagBag from /opt/airflow/dags/duckdb_example.py
INFO - Executing cmd: uv venv --allow-existing --seed --python python /opt/airflow/tmp/venvizdpi6sh
INFO - Output:
INFO - Using CPython 3.12.11 interpreter at: /usr/python/bin/python
INFO - Creating virtual environment with seed packages at: tmp/venvizdpi6sh
INFO -  + pip==25.2
INFO - Executing cmd: uv pip install --python /opt/airflow/tmp/venvizdpi6sh/bin/python -r /opt/airflow/tmp/venvizdpi6sh/requirements.txt
INFO - Output:
INFO - Resolved 7 packages in 167ms
INFO - Installed 7 packages in 3.50s
INFO -  + duckdb==1.1.1
INFO -  + numpy==2.3.0
INFO -  + pandas==2.3.2
INFO -  + python-dateutil==2.9.0.post0
INFO -  + pytz==2025.2
INFO -  + six==1.17.0
INFO -  + tzdata==2025.2
INFO - Executing cmd: /opt/airflow/tmp/venvizdpi6sh/bin/python /opt/airflow/tmp/venv-callm_tfnrbj/script.py /opt/airflow/tmp/venv-callm_tfnrbj/script.in /opt/airflow/tmp/venv-callm_tfnrbj/script.out /opt/airflow/tmp/venv-callm_tfnrbj/string_args.txt /opt/airflow/tmp/venv-callm_tfnrbj/termination.log /opt/airflow/tmp/venv-callm_tfnrbj/airflow_context.json
INFO - Output:
INFO - Query result:
INFO -     answer
INFO - 0      42
WARNING - Fail to delete /opt/airflow/opt/airflow/tmp/venvizdpi6sh. The directory does not exist.
INFO - Done. Returned value was: None

What you think should happen instead?

Airflow should look for the venv in the same directory where it is created instead of always prefixing the dir with /opt/airflow/ when it's time to delete the venv.

INFO - Deleted venv /opt/airflow/tmp/venvizdpi6sh.

How to reproduce

Below is the example DAG that I am running, Along with a snippet of the Docker Compose File that I have. The docker compose file is otherwise unchanged except for the fact that I removed flower and switched celery to LocalExecutor.

Example DAG

from airflow.sdk import DAG, task
import pendulum

with DAG(
    dag_id="duckdb_example",
    description="A simple dag to show the use of a python virtual env with duckdb",
    start_date=pendulum.datetime(2025, 8, 1, tz="EST"),
    schedule="@daily",
    catchup=False,
):
    @task.virtualenv(
        task_id="virtualenv_duckdb", 
        requirements=["duckdb==1.1.1", "numpy==2.3.0", "pandas==2.3.2"], 
        system_site_packages=False,
    )
    def run_query():
        import duckdb
        import time
        result = duckdb.query("SELECT 42 AS answer").to_df()
        print("Query result:\n", result)

    run_query()

Operating System

Linux - Ubuntu

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

I'm using Docker Compose with the below DockerFile.

FROM apache/airflow:3.1.0
ADD requirements.txt .
RUN pip install apache-airflow==${AIRFLOW_VERSION} -r requirements.txt

The docker compose file is based off the offical Airflow file. I switched celery for LocalExecutor since I'm just testing Airflow locally. The below section is what I mainly changed in the docker compose file. The only other things I chaged were removing parts of the file that celery would use such as Flower

x-airflow-common:
  &airflow-common
  # In order to add custom dependencies or upgrade provider distributions you can use your extended image.
  # Comment the image line, place your Dockerfile in the directory where you placed the docker-compose.yaml
  # and uncomment the "build" line below, Then run `docker-compose build` to build the images.
  # image: ${AIRFLOW_IMAGE_NAME:-apache/airflow:3.1.0}
  build: .
  environment:
    &airflow-common-env
    AIRFLOW__CORE__EXECUTOR: LocalExecutor
    AIRFLOW__CORE__AUTH_MANAGER: airflow.providers.fab.auth_manager.fab_auth_manager.FabAuthManager
    AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://airflow:airflow@postgres/airflow
    AIRFLOW__CELERY__RESULT_BACKEND: db+postgresql://airflow:airflow@postgres/airflow
    AIRFLOW__CELERY__BROKER_URL: redis://:@redis:6379/0
    AIRFLOW__CORE__FERNET_KEY: ''
    AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION: 'true'
    AIRFLOW__CORE__LOAD_EXAMPLES: 'false'
    AIRFLOW__CORE__EXECUTION_API_SERVER_URL: 'http://airflow-apiserver:8080/execution/'
    AIRFLOW__SCHEDULER__ENABLE_HEALTH_CHECK: 'true'
    # WARNING: Use _PIP_ADDITIONAL_REQUIREMENTS option ONLY for a quick checks
    # for other purpose (development, test and especially production usage) build/extend Airflow image.
    _PIP_ADDITIONAL_REQUIREMENTS: ${_PIP_ADDITIONAL_REQUIREMENTS:-}
    AIRFLOW_CONFIG: '/opt/airflow/config/airflow.cfg'
    # Set temp directory to use mounted volume
    TMPDIR: '/opt/airflow/tmp'
    UV_CACHE_DIR: '/opt/airflow/tmp'
    UV_LINK_MODE: 'copy'
  volumes:
    - ${AIRFLOW_PROJ_DIR:-.}/dags:/opt/airflow/dags
    - ${AIRFLOW_PROJ_DIR:-.}/logs:/opt/airflow/logs
    - ${AIRFLOW_PROJ_DIR:-.}/config:/opt/airflow/config
    - ${AIRFLOW_PROJ_DIR:-.}/plugins:/opt/airflow/plugins
  user: "${AIRFLOW_UID:-50000}:0"
  depends_on:
    &airflow-common-depends-on
    redis:
      condition: service_healthy
    postgres:
      condition: service_healthy

Anything else?

This issue occurs with every PythonVirtualenvOperator DAG I've tried recently

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[olegkachur-e]

Hi @RaphCodec!

The warning you see is not about venv dir itself, the venv dirs cleaned up automatically, as they are created as TempDir (if no cache). I don't think we need a log.info message about that.

There was an issue (#53390) with pycache files trace, on some circumstances were left on paths like this /opt/airflow/opt/airflow/tmp/venvizdpi6sh, so I added the additional clean up step.

In your case there are no such traces left, so there is actually nothing to delete and it can be ignored.

[RaphCodec]

Oh I get it now. I misunderstood the warning then. Thank you for the clarification @olegkachur-e ! Much appreciated.