Investigate option for execution API Connections datamodel for connections with no `conn_type`

[amoghrajesh]

Body

When a Connection has conn_type=None (which is valid for secrets backend connections), ConnectionResponse.model_validate(connection) fails because the Pydantic model requires conn_type: str (not optional).

TLDR:

1. Secrets backends can return connections without conn_type (backward compatible with Airflow 2)
2. The database model requires conn_type (nullable=False), but secrets backend connections are ephemeral and don't go through the database.

Qns that need investigation:

1. Should ConnectionResponse models allow conn_type: str | None to support secrets backend connections?
2. What's the impact on API consumers (UI, CLI, other clients) if we make conn_type optional?

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[jroachgolf84]

Just my two cents; I think there are a couple of scenarios where this could be an issue:

• Silent-ish failures in Operators/Sensors/Hooks when trying to use connections that don't have a conn_type. Maybe there is explicit error handling, but I haven't checked it out.
• Inability for users to port one-for-one from a Secrets Backend to storing connections in the metadata DB (I can see this being a common issue).
• Unintentional mis-use of Connections (maybe a user should be using a Variable rather than a Connection).

I think requiring conn_type provides a more unified experience for users across the board.

[jroachgolf84]

@amoghrajesh, I've done a bit of thinking here, as well as chatted with folks moving from Airflow 2 -> 3. This is a breaking change for folks who are migrating. In fact, they may have to go through hundreds/thousands of Secrets/Variables to add a conn_type. I'd suggest allowing conn_type: str | None.