Skip to content

Azure remote logging config remote_base_log_folder not properly working #58946

@alessandro-peyrachia-rnc

Description

Apache Airflow Provider(s)

microsoft-azure

Versions of Apache Airflow Providers

12.8.1

Apache Airflow version

3.1.3

Operating System

Ubuntu 22.04

Deployment

Official Apache Airflow Helm Chart

Deployment details

Deployed in AKS 1.32.7
Helm Chart 1.18.0 with airflow image 3.1.3 + updated apache-airflow-providers-microsoft-azure@12.8.1

Remote logging configuration

AIRFLOW__LOGGING__REMOTE_LOGGING=true
AIRFLOW__LOGGING__REMOTE_LOG_CONN_ID=azure_default_logging #custom connection
AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER=wasb://logs 
AIRFLOW__AZURE_REMOTE_LOGGING__REMOTE_WASB_LOG_CONTAINER=airflow-logs

What happened

Checking the documentation, configuration AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER should start with wasb:// and the folder where to store logs in the designated container.

By setting it as described above, logs are stored in airflow-logs/wasb:/{No Name}/logs

What you think should happen instead

Log output is working
Image

Log path is a little off

Image

How to reproduce

simply set AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER=wasb://logs

Anything else

As a workaround, setting
AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER=wasb-logs
correctly stores the log in the wasb-logs folder at the root of the container

Image

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions