Test connection always fails in airflow ui and cli

[choo121600]

Apache Airflow version

main (development)

If "Other Airflow 2/3 version" selected, which one?

No response

What happened?

When I click “Test connection” in the Airflow UI and CLI, the test always fails.

However, when running an actual Dag that uses the same connection, it works perfectly fine.

What you think should happen instead?

No response

How to reproduce

1. Go to Admin --> Connections
2. Select any valid connection
3. Click “Test”

Operating System

osx

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[merlinux]

Same problem here.
1) From CLI, I can get the conn, but not test it.

(venv-py312) [airflow@air3d01xdl ~]$ airflow connections get rac8_world_airflow

(venv-py312) [airflow@air3d01xdl ~]$ airflow connections test rac8_world_airflow

2) From UI, looks like SERVICE_NAME is missed or wrong parsed.

Thanks a lot.

[nzbhada]

Same problem on latest 3.1.5 version

[choo121600]

It seems related to changes in sensitive data management in AF3. (I haven’t had look at it in detail yet.)

The actual Dags and Connections work fine; the issue only affects "test connection." so I think it’s okay to address this gradually. I’ll likely take a look and try to resolve it after my vacation :)

[merlinux]

The actual Dags and Connections work fine; the issue only affects "test connection."

In my case, Dags and Connections are unusable. For example:

I also checked an Oracle Provider conn, wich returns "ORA-12504: TNS:listener was not given the SERVICE_NAME in CONNECT_DATA" in test mode and when I'm using the hook in a DAG (like is showed in my above capture)

BTW: note I'm not saying connections are untesteable. For me, connections are unusable, verifyed on PostgreSLQ and Oracle.

Glad to send you more specific evidence, just say it.
Thanks in advance!

[choo121600]

BTW: note I'm not saying connections are untesteable. For me, connections are unusable, verifyed on PostgreSLQ and Oracle.

Hmm, that's interesting🤔
I haven't seen any major issues with the provider I'm using. I'll need to take a closer look.

Could you let me know which versiobn of Airflow and the provider you're using?

[merlinux]

Could you let me know which versiobn of Airflow and the provider you're using?

Using Airflow 3.1.5, and the installed providers are:

Also using thick client extra json conf with Oracle Prov and instantclient_23_26, which runs OK from command line but failing from connections with the mentioned error (not given the SERVICE_NAME)
Cheers!

[choo121600]

After looking into the issue, it seems that the CLI and the UI have different causes.

The issue I found is in the connections_get and connections_test CLI commands. These commands were not setting the _AIRFLOW_PROCESS_CONTEXT=server environment variable.
This variable is used by the Secrets Backend system to decide whether to use the MetastoreBackend (server context) or another backend (client context).

Because the process context was not set correctly, the CLI ended up using the wrong Secrets Backend.
In the CLI, this issue can be fixed by setting _AIRFLOW_PROCESS_CONTEXT=server.

I created PR #59624 to fix this issue.

For the UI, this looks like a separate problem related to masking("***"), so it would be better to handle it in a different PR.
I plan to work on that as a follow-up.