-
Notifications
You must be signed in to change notification settings - Fork 16.4k
postgres provider: use non-binary psycopg2 #25710
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
|
362d42f to
d3ebcc7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this file is generated automatically so you shouldn't edit it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@RosterIn is right. see note just abiive:
Please, only add notes to the Changelog just below the "Changelog" header when there are some breaking changes
and you want to add an explanation to the users on how they are supposed to deal with them.
The changelog is updated and maintained semi-automatically by release manager.
Also I think the change is not breaking. Dependency change is not a breaking change, bumping min version is not breaking either, and the behaviour is not going to change either
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @RosterIn , I've removed the changes to index.rst.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@potiuk I considered it a breaking change because it will break for some people, because it introduces new operating system dependencies, that were not necessarily present before. For example on Debian-based distros, libpq-dev is necessary. This is the case for the slim docker images, for example airflow:slim-2.3.3-python3.10 .
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not what we consider as breaking. Dependency changes are not introducing. Our SemVer rules state (see readme)
Airflow: SemVer rules apply to core airflow only (excludes any changes to providers). Changing limits for versions of Airflow dependencies is not a breaking change on its own.
And while this is not changing the version but other 'variant' of the same dependency, this only affect how you install airflow. And it is not breaking the 'use' of Airflow which is the important factor for breaking/non breaking decision.
The main thing here that changes that are only affecting the way how you install Airfloe are not really 'breaking' any user code or ways how you use airflow. If for any reason you cannot install Airflow new version, you just cannot install it and you don't break anything. Once you installed it, it works without changes. So it does not break anything.
If you follow the logic of 'dependencies are breaking changes' then it basically would mean than any release where any dependency change is breaking - because it might cause troubles when installing Airflow or providers.
|
OK. Thanks for the explanation! I'll adjust it later.
…On Sat, 20 Aug 2022, 11:22 Jarek Potiuk ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In docs/apache-airflow-providers-postgres/index.rst
<#25710 (comment)>:
> @@ -65,7 +65,7 @@ Package apache-airflow-providers-postgres
`PostgreSQL <https://www.postgresql.org/>`__
-Release: 5.2.0
+Release: 6.0.0
This is not what we consider as breaking. Dependency changes are not
introducing. Our SemVer rules state (see readme)
Airflow: SemVer rules apply to core airflow only (excludes any changes to
providers). Changing limits for versions of Airflow dependencies is not a
breaking change on its own.
And while this is not changing the version but other 'variant' of the same
dependency, this only affect how you install airflow. And it is not
breaking the 'use' of Airflow which is the important factor for
breaking/non breaking decision.
The main thing here that changes that are only affecting the way how you
install Airfloe are not really 'breaking' any user code or ways how you use
airflow. If for any reason you cannot install Airflow new version, you just
cannot install it and you don't break anything. Once you installed it, it
works without changes. So it does not break anything.
If you follow the logic of 'dependencies are breaking changed' then it
basically would mean than any release where any dependency change is
breaking - because it might cause troubles when installing Airflow or
providers.
—
Reply to this email directly, view it on GitHub
<#25710 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAFKHGHUKE2TMV6P6DHE6ALV2CPUTANCNFSM56PS4F6A>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
60bafca to
868a8b0
Compare
|
Cool! |
|
Awesome work, congrats on your first merged pull request! |
The psycopg dependency is slightly more optimized than than the binary one, but the problems it creates when it is used as dependency are numerous - it always expects to compile the psycopg dependencies, so you cannot install it when there are no build-essentials installed and on some systems additional libraries are needed. We introduced it in apache#25710 for optimisation reasons. However, the optimisations it brings are not really justified with the installation complexities it introduces, that's why we revert it.
The psycopg dependency is slightly more optimized than than the binary one, but the problems it creates when it is used as dependency are numerous - it always expects to compile the psycopg dependencies, so you cannot install it when there are no build-essentials installed and on some systems additional libraries are needed. We introduced it in #25710 for optimisation reasons. However, the optimisations it brings are not really justified with the installation complexities it introduces, that's why we revert it.
The non-binary version of psycopg2 is now used, and the version requirement changed to psycopg2>=2.8.0.
The non-binary version is used because the binary version of psycopg2 is not recommended for production
use. The minimum version of psycopg2 was changed because psycopg2 < 2.8.0 includes a binary distribution
which will be installed by default.
If not already present, you will need to add the postgresql development libraries. On debian-based systems,
this is provided by the package libpq-dev.
Closes: #25712
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.