Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate hql parameters and synchronize DBApiHook method APIs #25299

Merged
merged 2 commits into from
Jul 27, 2022

Conversation

potiuk
Copy link
Member

@potiuk potiuk commented Jul 26, 2022

Various providers deriving from DbApi had some variations in some
methods that were derived from the common DbApi Hook. Mostly they
were about extra parameters added and hql parameter used instead of
sql. This prevents from really "common" approach in DbApiHook as
some common sql operators rely on signatures being the same.

This introduced breaking changes in a few providers - but those
breaking changes are easy to fix and most have already been
deprecated.

I also had to make sure all types are correct (hence cloud_sql typing fixes).


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@potiuk potiuk requested a review from turbaszek as a code owner July 26, 2022 08:42
@boring-cyborg boring-cyborg bot added area:providers provider:Apache provider:google Google (including GCP) related issues labels Jul 26, 2022
@potiuk potiuk requested a review from eladkal July 26, 2022 08:42
@potiuk
Copy link
Member Author

potiuk commented Jul 26, 2022

cc: @kazanzhy as well. In order to implement #25293 I needed to synchronize all methods in DbAPIHook derived providers (breaking changes in some operators removing deprecated usages). That should make it easier to unify some of the code you plan.

@potiuk
Copy link
Member Author

potiuk commented Jul 26, 2022

This also simplifies and removes some MyPy Hacks with "overloads" :)

@potiuk potiuk force-pushed the synchronize-signatures-between-dbapi-hooks branch from d7f4831 to d9dffc8 Compare July 26, 2022 08:54
@potiuk potiuk force-pushed the synchronize-signatures-between-dbapi-hooks branch from d9dffc8 to 83c6726 Compare July 26, 2022 09:02
@potiuk potiuk force-pushed the synchronize-signatures-between-dbapi-hooks branch from 83c6726 to fd331e0 Compare July 26, 2022 09:53
@potiuk potiuk requested review from ashb and mik-laj as code owners July 26, 2022 09:53
@potiuk
Copy link
Member Author

potiuk commented Jul 26, 2022

Part of this one was extracted to #25301 - let's see if this one succeds, if so, then we can merge #25301 first to separate the change logically.

@potiuk potiuk force-pushed the synchronize-signatures-between-dbapi-hooks branch 3 times, most recently from 7b851fc to b2a3329 Compare July 26, 2022 11:34
@potiuk potiuk force-pushed the synchronize-signatures-between-dbapi-hooks branch from b2a3329 to 3d23031 Compare July 27, 2022 10:15
@potiuk
Copy link
Member Author

potiuk commented Jul 27, 2022

All parameters updated @kazanzhy :)

@potiuk potiuk force-pushed the synchronize-signatures-between-dbapi-hooks branch 2 times, most recently from 8567e08 to 40c94f9 Compare July 27, 2022 12:08
@potiuk
Copy link
Member Author

potiuk commented Jul 27, 2022

Should be green now.

Various providers deriving from DbApi had some variations in some
methods that were derived from the common DbApi Hook. Mostly they
were about extra parameters added and hql parameter used instead of
sql. This prevents from really "common" approach in DbApiHook as
some common sql operators rely on signatures being the same.

This introduced breaking changes in a few providers - but those
breaking changes are easy to fix and most have already been
deprecated.
@potiuk potiuk force-pushed the synchronize-signatures-between-dbapi-hooks branch from 40c94f9 to 7a98394 Compare July 27, 2022 12:49
@potiuk
Copy link
Member Author

potiuk commented Jul 27, 2022

Should be merged after #25338 (this PR incorporates change that should be separated).

@potiuk
Copy link
Member Author

potiuk commented Jul 27, 2022

(now it should be green :) )

@potiuk
Copy link
Member Author

potiuk commented Jul 27, 2022

Green :)

@@ -143,7 +143,7 @@ function testing::setup_docker_compose_backend() {
# so we need to mount an external volume for its db location
# the external db must allow for parallel testing so TEST_TYPE
# is added to the volume name
export MSSQL_DATA_VOLUME="${HOME}/tmp-mssql-volume-${TEST_TYPE}-${MSSQL_VERSION}"
export MSSQL_DATA_VOLUME="${HOME}/tmp-mssql-volume-${TEST_TYPE/\[*\]/}-${MSSQL_VERSION}"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already merged in main via #25338

Co-authored-by: eladkal <45845474+eladkal@users.noreply.github.com>
@potiuk
Copy link
Member Author

potiuk commented Jul 27, 2022

Woohooo!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers provider:google Google (including GCP) related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants