Connection id "extra" error

[Dobring]

Apache Airflow version

Other Airflow 2 version (please specify below)

What happened

Dear Sir/Madam.
In airflow 2.7.0 version
After editing connection in GUI,
Field extra recieves default value of {}, which is not equal to None.

This causes extra={} argument being passed to connection uri.
{} is encoded to %7B%7D, as you can see on screen attached.

The result of default value is attached
It leads to the following error message.

sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) invalid dsn: invalid connection option "extra"
(Background on this error at: https://sqlalche.me/e/14/f405)

What you think should happen instead

I expect that if I leave Extra field empty when creating a connection id, there shall no be extra information.

How to reproduce

We simply create the connection by manually inputing all usual credentials.

Operating System

Linux

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==8.5.1
apache-airflow-providers-postgres==5.6.0

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[slavanorm]

See PR being reason for the issue

#28583
#32309

[Taragolis]

@Dobring could you provide a bit more information. It's quite difficult to understand what is going wrong here.

1. Which version of Amazon and Postgres provider are you use
2. Full traceback of error
3. Minimal step for reproduce, please provide it as text so someone who would check might replace by it owns values. ⚠️⚠️ You don't need to provide your actual credentials, just samples ⚠️⚠️

[Dobring]

Dear Taragolis, please find the full traceback of error I have received,
dag_id=prod-test_run_id=manual__2023-09-18T20_23_39.862391+00_00_task_id=print_player_attempt=1.log

[Taragolis]

Ok, now seems I got what a problem.
Last changes finally break PostgresHook.get_uri logic, to be honest it was broken for a long time, maybe even since it implemented.

The problem with bad assumption that Airflow Connection URI is the same as psycopg2 or even sqlalchemy URI. But it not the same, and was never the same due to the fact that Airflow PostgreSQL connection has extra attributes which is not supported by psycopg2/SQLAlcemy and current implementation takes Airflow URI and replaced schema. So it is need to be fixed in PostgreSQL provider and maybe others.

[slavanorm]

so, we could fix it in our case by manually going to connections GUI, deleting {} in extra field, and saving connection.
im pretty sure if you edit connection without removing default {} value in extra field, it will ruin connection again.

[Taragolis]

so, we could fix it in our case by manually going to connections GUI, deleting {} in extra field, and saving connection.

As temporary fix it might help

ruin connection again.

The problem not in Airflow Connection itself. The problem in PostgresHook implementation because Airflow Connection != SQLAlchemy.

[github-actions]

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

[github-actions]

This issue has been closed because it has not received response from the issue author.