Skip to content

Conversation

@amoghrajesh
Copy link
Contributor

When connection / secrets backend isnt setup, we should not log that those entities aren't found there.
It leads to wrong information and confusion.

DAG used:

from __future__ import annotations

from airflow.models.baseoperator import BaseOperator
from airflow import DAG
from airflow.sdk.definitions.connection import Connection


class CustomOperator(BaseOperator):
    def execute(self, context):
        gc = Connection.get("athena_default")
        print("The conn is", gc)
        # print("Conn uri is", gc.get_uri())


with DAG("example_get_connection_from_defn", schedule=None, catchup=False) as dag:
    CustomOperator(task_id="print_conn")

No backend configured, logs show:

[2025-06-16, 17:55:11] INFO - 2025-06-16 12:25:11 [debug    ] DAG file parsed                [task] file=example-metadatadb-backend.py: chan="stdout": source="task"
[2025-06-16, 17:55:11] INFO - 2025-06-16 12:25:11 [debug    ] Connection not found in any of the configured Secrets Backends. Trying to retrieve from API server [task] conn_id=athena_default: chan="stdout": source="task"
[2025-06-16, 17:55:[11](http://localhost:28080/dags/example_get_connection_from_defn/runs/manual__2025-06-16T12:25:06.782393+00:00/tasks/print_conn?try_number=1#11)] INFO - The conn is Connection(conn_id='athena_default', conn_type='athena', description=None, host=None, schema=None, login=None, password=None, port=None, extra=None): chan="stdout": source="task"

After fix:

DAG bundles loaded: dags-folder: source="airflow.dag_processing.bundles.manager.DagBundlesManager"
[2025-06-16, 18:06:[3](http://localhost:28080/dags/example_get_connection_from_defn/runs/manual__2025-06-16T12:36:33.447212+00:00/tasks/print_conn?try_number=1#3)3] INFO - Filling up the DagBag from /files/dags/example-metadatadb-backend.py: source="airflow.models.dagbag.DagBag"
[2025-06-16, 18:06:33] INFO - The conn is Connection(conn_id='athena_default', conn_type='athena', description=None, host=None, schema=None, login=None, password=None, port=None, extra=None): chan="stdout": source="task"

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@amoghrajesh amoghrajesh requested review from ashb and kaxil as code owners June 16, 2025 12:37
@amoghrajesh amoghrajesh added the backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch label Jun 16, 2025
Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com>
@amoghrajesh amoghrajesh merged commit 23ecee8 into apache:main Jun 17, 2025
71 checks passed
@amoghrajesh amoghrajesh deleted the do-not-log-for-backends branch June 17, 2025 08:32
github-actions bot pushed a commit that referenced this pull request Jun 17, 2025
…riables (#51786)

(cherry picked from commit 23ecee8)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
@github-actions
Copy link

Backport successfully created: v3-0-test

Status Branch Result
v3-0-test PR Link

RoyLee1224 pushed a commit to RoyLee1224/airflow that referenced this pull request Jun 21, 2025
kaxil pushed a commit that referenced this pull request Jun 30, 2025
…riables (#51786) (#51826)

(cherry picked from commit 23ecee8)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:task-sdk backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants