Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Poetry #1682

Merged
merged 1 commit into from
Nov 6, 2024
Merged

Add support for Poetry #1682

merged 1 commit into from
Nov 6, 2024

Conversation

edmorley
Copy link
Member

@edmorley edmorley commented Oct 31, 2024

After many refactoring/preparation PRs, we're now ready to add support for the package manager Poetry 🎉
https://python-poetry.org

To use Poetry, apps must have a poetry.lock lockfile, which can be created by running poetry lock locally, after adding Poetry config to pyproject.toml (which can be done either manually or by using poetry init).

For now, if a requirements.txt or Pipfile is found it will take precedence over poetry.lock for backwards compatibility (in the future this will become a warning then an error). This means users of the third-party python-poetry-buildpack will need to remove that buildpack in order to use the new native Poetry support, since it exports a requirements.txt file during the build.

Poetry is installed into the build cache rather than the slug, so is not available at run-time (since it's not typically needed at run-time and doing so reduces the slug size). The entrypoints of installed dependencies are available on PATH, so use of poetry run or poetry shell is not required at run-time to use dependencies in the environment.

When using Poetry, pip is not installed since Poetry includes its own internal vendored copy that it will use instead (for the small number of Poetry operations for which it still calls out to pip, such as package uninstalls).

During normal (non-CI) builds, the poetry install --sync command is run using --only main so as to only install the main [tool.poetry.dependencies] dependencies group from pyproject.toml and not any of the app's other dependency groups (such as test/dev/... groups, eg [tool.poetry.group.test.dependencies]).

On Heroku CI, all default Poetry dependency groups are installed (i.e. all groups minus those marked as optional = true).

Relevant Poetry docs:

See also the Python CNB equivalent of this PR:

Note: We don't support controlling the Python version via Poetry's tool.poetry.dependencies.python field, since that field typically contains a version range, which is not safe to use. Use the newly added .python-version file support instead. For more on this, see the longer explanation over in the Python CNB repo:
heroku/buildpacks-python#260

Closes #796.
Closes #835.
GUS-W-16810914.

@edmorley edmorley self-assigned this Oct 31, 2024
@edmorley edmorley marked this pull request as ready for review October 31, 2024 22:00
@edmorley edmorley requested a review from a team as a code owner October 31, 2024 22:00
After many refactoring/preparation PRs, we're now ready to add support
for the package manager Poetry:
https://python-poetry.org

To use Poetry, apps must have a `poetry.lock` lockfile, which can be
created by running `poetry lock` locally, after adding Poetry config to
`pyproject.toml` (which can be done either manually or by using
`poetry init`).

For now, if a `requirements.txt` or `Pipfile` is found it will take
precedence over `poetry.lock` for backwards compatibility (in the future
this will become a warning then an error). This means users of the
third-party `python-poetry-buildpack` will need to remove that buildpack
in order to use the new native Poetry support, since it exports a
`requirements.txt` file during the build.

Poetry is installed into the build cache rather than the slug, so is not
available at run-time (since it's not typically needed at run-time and
doing so reduces the slug size). The entrypoints of installed
dependencies are available on `PATH`, so use of `poetry run` or
`poetry shell` is not required at run-time to use dependencies in the
environment.

When using Poetry, pip is not installed since Poetry includes its own
internal vendored copy that it will use instead (for the small number
of Poetry operations for which it still calls out to pip, such as
package uninstalls).

During normal (non-CI) builds, the `poetry install --sync` command is
run using `--only main` so as to only install the main
`[tool.poetry.dependencies]` dependencies group from `pyproject.toml`
and not any of the app's other dependency groups (such as test/dev/...
groups, eg `[tool.poetry.group.test.dependencies]`).

On Heroku CI, all default Poetry dependency groups are installed (i.e.
all groups minus those marked as `optional = true`).

Relevant Poetry docs:
- https://python-poetry.org/docs/cli/#install
- https://python-poetry.org/docs/configuration/
- https://python-poetry.org/docs/managing-dependencies/#dependency-groups

See also the Python CNB equivalent of this PR:
- heroku/buildpacks-python#261

Note: We don't support controlling the Python version via Poetry's
`tool.poetry.dependencies.python` field, since that field typically
contains a version range, which is not safe to use. Use the newly
added `.python-version` file support instead. For more on this, see
the longer explanation over in the Python CNB repo:
heroku/buildpacks-python#260

Refs #796.
GUS-W-16810914.
@edmorley edmorley merged commit 895a004 into main Nov 6, 2024
7 checks passed
@edmorley edmorley deleted the poetry branch November 6, 2024 09:12
@heroku-linguist heroku-linguist bot mentioned this pull request Nov 6, 2024
This was referenced Nov 6, 2024
@edmorley
Copy link
Member Author

edmorley commented Nov 6, 2024

edmorley added a commit that referenced this pull request Nov 6, 2024
The existing Poetry bootstrap process added in #1682 used the pip wheel
bundled within the Python stdlib.

This required use of pip's `--python` option, which was added to pip in
v22.3 in 2022. All of the major Python versions we support have been
updated to that pip version or newer, the older patch releases of some
of those major Python versions can contain pip versions that are older.

Whilst we strongly recommend upgrading to newer patch releases (since
older versions are missing security updates and so likely insecure),
we still want to support using Poetry on these versions, so the I've
adjusted the bootstrap process to no longer use `--python`.
edmorley added a commit that referenced this pull request Nov 6, 2024
The existing Poetry bootstrap process added in #1682 used the pip wheel
bundled within the Python stdlib.

This required use of pip's `--python` option, which was added to pip in
v22.3 in 2022. All of the major Python versions we support have been
updated to that pip version or newer, however, the older patch releases
of some of those major Python versions can contain pip versions that are
older (for example, whilst latest Python 3.9.x bundles pip v23.0.1,
Python 3.9.0 bundles pip v20.2.1).

Previously, using those older patch versions would result in:

```
-----> Installing Poetry 1.8.4

Usage:   
  pip <command> [options]

no such option: --python

 !     Error: Unable to install Poetry.
```

Whilst we strongly recommend users upgrade to newer patch releases
(since older versions are missing security updates and so likely
insecure), we still want to support using Poetry on these versions, so
the I've adjusted the bootstrap process to no longer use `--python`
to prevent that error.
edmorley added a commit to edmorley/python-poetry-buildpack that referenced this pull request Nov 18, 2024
This buildpack is no longer required now that the Heroku Python
buildpack supports Poetry natively:
- https://devcenter.heroku.com/changelog-items/3050
- heroku/heroku-buildpack-python#1682

As such, a deprecation warning has been added to both the build
log and the README.

The warning borrows from the implementation here (which uses
colour, and also makes sure to ANSI wrap each line individually
to prevent issues with streaming logs from `git push` etc):
https://github.com/heroku/heroku-buildpack-python/blob/ca99c39f9e3627fff727788206f7fcc4ff064b7f/lib/output.sh#L50-L66

Closes moneymeets#75.
edmorley added a commit to edmorley/python-poetry-buildpack that referenced this pull request Nov 18, 2024
This buildpack is no longer required now that the Heroku Python
buildpack supports Poetry natively:
- https://devcenter.heroku.com/changelog-items/3050
- heroku/heroku-buildpack-python#1682

As such, a deprecation warning has been added to both the build
log and the README.

The warning borrows from the implementation here (which uses
colour, and also makes sure to ANSI wrap each line individually
to prevent issues with streaming logs from `git push` etc):
https://github.com/heroku/heroku-buildpack-python/blob/ca99c39f9e3627fff727788206f7fcc4ff064b7f/lib/output.sh#L50-L66

Closes moneymeets#75.
edmorley added a commit to edmorley/python-poetry-buildpack that referenced this pull request Nov 18, 2024
…ired

This buildpack is no longer required now that the Heroku Python
buildpack supports Poetry natively:
- https://devcenter.heroku.com/changelog-items/3050
- heroku/heroku-buildpack-python#1682

As such, a deprecation warning has been added to both the build
log and the README.

The warning borrows from the implementation here (which uses
colour, and also makes sure to ANSI wrap each line individually
to prevent issues with streaming logs from `git push` etc):
https://github.com/heroku/heroku-buildpack-python/blob/ca99c39f9e3627fff727788206f7fcc4ff064b7f/lib/output.sh#L50-L66

Closes moneymeets#75.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for Poetry
2 participants