Replace API server’s direct Connection access workaround in BaseHook

[amoghrajesh]

closes: #52520

During the migration of BaseHook to task sdk, I introduced a temporary workaround in the API server context:
https://github.com/apache/airflow/pull/51873/files#diff-acfa34467241916c1189174e98f87a4094e8cbf58d391fa084e6afb2594decbcR61-R71 to help solve some compat issues which arrived along the way.

When in the API server context, the system uses the Connection model (via custom backend > env var > DB fallback) directly.

This issue was on hold because it was dependent on coupling between Git bundle and API server. For view_url use case, a slim git bundle is instantiated to form the view_url leading to it using BaseHook to get a connection and BaseHook uses Connection from SDK, it goes back to API server causing a self loop scenario.

That has been fixed via: #52876 so attempting the removal again

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

[amoghrajesh]

Created the PR, let's see what tests start to fail.

[amoghrajesh]

Core failures are handled by #54084

[amoghrajesh]

Alright, fighting with the CI for compat + provider tests!

[amoghrajesh]

Alright, kube tests should be fixed by: #54261

Last set of compat failing!

[amoghrajesh]

@ashb I made updates on this one, could you take another look when possible?

[amoghrajesh]

Failing K8s tests to be fixed by #54385

[potiuk]

Nice one!

[amoghrajesh]

@seanghaeli the eventual goal is to move towards using the task sdk for these sort of things, so I'd suggest that the task sdk is installed

[potiuk]

@seanghaeli the eventual goal is to move towards using the task sdk for these sort of things, so I'd suggest that the task sdk is installed

Ah you are right :)

[potiuk]

But also @amoghrajesh -> we should handle the case where system tests are run on Airflow 2 -> I guess we cannot install task.sdk on 2 and we will never aim to do it ?

[amoghrajesh]

@potiuk @o-nikolas I do not think that should be an issue at all.

For system tests running on Airflow 2, we use the BaseHook class + deps from Airflow 2 itself. In other words, the BaseHook doesnt come from task sdk in those cases.

Most hook imports for amazon systests are from base_aws, and see the import for BaseHook there: https://github.com/apache/airflow/blob/main/providers/amazon/src/airflow/providers/amazon/aws/hooks/base_aws.py#L63 (its from a compat layer)