Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for PEP 735: Dependency Groups in pyproject.toml #8090

Open
zanieb opened this issue Oct 10, 2024 · 15 comments · May be fixed by #8272
Open

Support for PEP 735: Dependency Groups in pyproject.toml #8090

zanieb opened this issue Oct 10, 2024 · 15 comments · May be fixed by #8272
Labels
compatibility Compatibility with a specification or another tool enhancement New feature or request

Comments

@zanieb
Copy link
Member

zanieb commented Oct 10, 2024

This PEP was just accepted — we plan to support it. We'll track it here.

The implementation can be tracked in #8272

@zanieb zanieb added enhancement New feature or request compatibility Compatibility with a specification or another tool labels Oct 10, 2024
@zanieb zanieb changed the title Support for PEP 735 Support for PEP 735: Dependency Groups in pyproject.toml Oct 10, 2024
@NeilGirdhar
Copy link

NeilGirdhar commented Oct 12, 2024

I'm just curious, but will there be a "dev" dependency group that supplants

[tool.uv]
dev-dependencies = …

or is it up to the user to specify the group= when running uv sync?

@henryiii
Copy link
Contributor

Dependency-groups don't allow self-dependencies, so dev-dependencies = ["<self>[test]"] wouldn't be replaceable with dependency groups. Also, I think/hope that dependency groups are going to feed into a new environment and task system. :)

@dschneiderch
Copy link

dschneiderch commented Oct 12, 2024 via email

@zanieb
Copy link
Member Author

zanieb commented Oct 12, 2024

Dependency-groups don't allow self-dependencies, so dev-dependencies = ["[test]"] wouldn't be replaceable with dependency groups.

Is this explicitly banned? I think group = ["my-project[test]"] might "just work" in uv because of how we treat package names that are shadowed by the project?

@zanieb
Copy link
Member Author

zanieb commented Oct 12, 2024

will there be a "dev" dependency group that supplants ...

I'm not sure yet. This is a significant open question I must address before adding support.

@zanieb
Copy link
Member Author

zanieb commented Oct 12, 2024

I noticed there is no mention of constraints in the pep...

I don't see any reason that we wouldn't apply constraints to the dependencies in the groups. Do you need a separate set of constraints?

@charliermarsh
Copy link
Member

I only see this section, but I don't see any other discussion in the finalized PEP of self-references of project dependencies in dependency groups.

@dschneiderch
Copy link

dschneiderch commented Oct 12, 2024

I noticed there is no mention of constraints in the pep...

I don't see any reason that we wouldn't apply constraints to the dependencies in the groups. Do you need a separate set of constraints?

I am asking about the equivalent of constraint-dependencies, which I don't see addressed. I have a similar use case as discussed #6518 . We use a managed airflow service. For dag development we only need a subset of the installed py dependencies and installing every py dependency is challenging and overkill because it requires several system dependencies. I'm slowly converting people to use uv, one of the motivations is having an easy, declarative approach to setting up a virtual environment for development.

here is an example from pyproject.toml from a repo that contains dags:

[tool.uv]
dev-dependencies = [apache-airflow[gcp,beam]==2.9.1]
constraint-dependencies = [ long list of transient dependencies from cloud composer docs]

cloud composer docs
I want to install what i need but I want to make sure I don't install a transient dependency with a different version from our prod environment.

1 difference from the other issue is we are not using the official airflow constraints file from the url (this is what i used to use before we moved to the managed airflow). Right now I am copying the list of installed packages into my pyproject.toml file.

@charliermarsh
Copy link
Member

@dschneiderch -- Apologies if I'm misunderstanding but I think constraint-dependencies will just continue to be the right thing to use there.

@dschneiderch
Copy link

dschneiderch commented Oct 12, 2024 via email

@jankatins
Copy link

Duplicates #5632 -> either this or the other one could probably be closed.

@potiuk
Copy link

potiuk commented Oct 16, 2024

Would love to get it in ... PR is coming I see :)

@kaxil
Copy link

kaxil commented Oct 16, 2024

Same, looking forward to using it in the Airflow repo

zanieb added a commit that referenced this issue Oct 16, 2024
Part of #8090 

As a basic first step, we parse these groups defined in `pyproject.toml`
files.
@zanieb zanieb linked a pull request Oct 16, 2024 that will close this issue
3 tasks
zanieb added a commit that referenced this issue Oct 16, 2024
Part of #8090 

Adds the ability to add and remove dependencies from arbitrary groups
using `uv add` and `uv remove`. Does not include resolving with the new
dependencies — tackling that in #8110.

Additionally, this does not yet resolve interactions with the existing
`dev` group — we'll tackle that separately as well. I probably won't
merge the stack until that design is resolved.
zanieb added a commit that referenced this issue Oct 16, 2024
… in `uv lock` (#8110)

Part of #8090 

Adds the ability to include a group (`--group`) in the sync or _only_
sync a group (`--only-group`). Includes all groups in the resolution,
which will have the same limitations as extras as described in #6981.

There's a great deal of refactoring of the "development" concept into
"groups" behind the scenes that I am continuing to defer here to
minimize the diff.

Additionally, this does not yet resolve interactions with the existing
`dev` group — we'll tackle that separately as well. I probably won't
merge the stack until that design is resolved. The current proposal is
that we'll just "combine' the `dev-dependencies` contents into the `dev`
group.
zanieb added a commit that referenced this issue Oct 18, 2024
Part of #8090 
Unblocks #8274

Refactors `DevMode` and `DevSpecification` into a shared type
`DevGroupsSpecification` that allows us to track if `--dev` was
implicitly or explicitly provided.
zanieb added a commit that referenced this issue Oct 18, 2024
zanieb added a commit that referenced this issue Oct 18, 2024
Part of #8090 

Most of the heavy lifting is done in #8309
charliermarsh pushed a commit that referenced this issue Oct 18, 2024
…8338)

Part of #8090 

Most of the heavy lifting is done in #8309 

Includes `--only-dev` which appears to be missing as an oversight.
charliermarsh pushed a commit that referenced this issue Oct 20, 2024
Part of #8090

As a basic first step, we parse these groups defined in `pyproject.toml`
files.
charliermarsh pushed a commit that referenced this issue Oct 20, 2024
Part of #8090

Adds the ability to add and remove dependencies from arbitrary groups
using `uv add` and `uv remove`. Does not include resolving with the new
dependencies — tackling that in #8110.

Additionally, this does not yet resolve interactions with the existing
`dev` group — we'll tackle that separately as well. I probably won't
merge the stack until that design is resolved.
charliermarsh pushed a commit that referenced this issue Oct 20, 2024
… in `uv lock` (#8110)

Part of #8090

Adds the ability to include a group (`--group`) in the sync or _only_
sync a group (`--only-group`). Includes all groups in the resolution,
which will have the same limitations as extras as described in #6981.

There's a great deal of refactoring of the "development" concept into
"groups" behind the scenes that I am continuing to defer here to
minimize the diff.

Additionally, this does not yet resolve interactions with the existing
`dev` group — we'll tackle that separately as well. I probably won't
merge the stack until that design is resolved. The current proposal is
that we'll just "combine' the `dev-dependencies` contents into the `dev`
group.
charliermarsh added a commit that referenced this issue Oct 20, 2024
…roups (#8266)

Part of #8090

Adds the ability to read group inclusions (`include-group = <name>`) in
the `pyproject.toml`. Resolves groups into concrete dependencies for
resolution.

See #8110 for a bit more commentary
on deferred work.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
charliermarsh pushed a commit that referenced this issue Oct 20, 2024
Part of #8090 
Unblocks #8274

Refactors `DevMode` and `DevSpecification` into a shared type
`DevGroupsSpecification` that allows us to track if `--dev` was
implicitly or explicitly provided.
charliermarsh pushed a commit that referenced this issue Oct 20, 2024
charliermarsh pushed a commit that referenced this issue Oct 20, 2024
Part of #8090

Most of the heavy lifting is done in #8309
charliermarsh pushed a commit that referenced this issue Oct 20, 2024
…8338)

Part of #8090

Most of the heavy lifting is done in #8309

Includes `--only-dev` which appears to be missing as an oversight.
charliermarsh pushed a commit that referenced this issue Oct 22, 2024
Part of #8090

As a basic first step, we parse these groups defined in `pyproject.toml`
files.
charliermarsh pushed a commit that referenced this issue Oct 22, 2024
Part of #8090

Adds the ability to add and remove dependencies from arbitrary groups
using `uv add` and `uv remove`. Does not include resolving with the new
dependencies — tackling that in #8110.

Additionally, this does not yet resolve interactions with the existing
`dev` group — we'll tackle that separately as well. I probably won't
merge the stack until that design is resolved.
charliermarsh pushed a commit that referenced this issue Oct 22, 2024
… in `uv lock` (#8110)

Part of #8090

Adds the ability to include a group (`--group`) in the sync or _only_
sync a group (`--only-group`). Includes all groups in the resolution,
which will have the same limitations as extras as described in #6981.

There's a great deal of refactoring of the "development" concept into
"groups" behind the scenes that I am continuing to defer here to
minimize the diff.

Additionally, this does not yet resolve interactions with the existing
`dev` group — we'll tackle that separately as well. I probably won't
merge the stack until that design is resolved. The current proposal is
that we'll just "combine' the `dev-dependencies` contents into the `dev`
group.
charliermarsh added a commit that referenced this issue Oct 22, 2024
…roups (#8266)

Part of #8090

Adds the ability to read group inclusions (`include-group = <name>`) in
the `pyproject.toml`. Resolves groups into concrete dependencies for
resolution.

See #8110 for a bit more commentary
on deferred work.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
charliermarsh pushed a commit that referenced this issue Oct 22, 2024
Part of #8090 
Unblocks #8274

Refactors `DevMode` and `DevSpecification` into a shared type
`DevGroupsSpecification` that allows us to track if `--dev` was
implicitly or explicitly provided.
charliermarsh pushed a commit that referenced this issue Oct 22, 2024
charliermarsh pushed a commit that referenced this issue Oct 22, 2024
Part of #8090

Most of the heavy lifting is done in #8309
charliermarsh pushed a commit that referenced this issue Oct 22, 2024
…8338)

Part of #8090

Most of the heavy lifting is done in #8309

Includes `--only-dev` which appears to be missing as an oversight.
charliermarsh pushed a commit that referenced this issue Oct 22, 2024
Part of #8090

As a basic first step, we parse these groups defined in `pyproject.toml`
files.
charliermarsh pushed a commit that referenced this issue Oct 22, 2024
Part of #8090

Adds the ability to add and remove dependencies from arbitrary groups
using `uv add` and `uv remove`. Does not include resolving with the new
dependencies — tackling that in #8110.

Additionally, this does not yet resolve interactions with the existing
`dev` group — we'll tackle that separately as well. I probably won't
merge the stack until that design is resolved.
charliermarsh pushed a commit that referenced this issue Oct 22, 2024
… in `uv lock` (#8110)

Part of #8090

Adds the ability to include a group (`--group`) in the sync or _only_
sync a group (`--only-group`). Includes all groups in the resolution,
which will have the same limitations as extras as described in #6981.

There's a great deal of refactoring of the "development" concept into
"groups" behind the scenes that I am continuing to defer here to
minimize the diff.

Additionally, this does not yet resolve interactions with the existing
`dev` group — we'll tackle that separately as well. I probably won't
merge the stack until that design is resolved. The current proposal is
that we'll just "combine' the `dev-dependencies` contents into the `dev`
group.
charliermarsh added a commit that referenced this issue Oct 22, 2024
…roups (#8266)

Part of #8090

Adds the ability to read group inclusions (`include-group = <name>`) in
the `pyproject.toml`. Resolves groups into concrete dependencies for
resolution.

See #8110 for a bit more commentary
on deferred work.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
charliermarsh pushed a commit that referenced this issue Oct 22, 2024
Part of #8090 
Unblocks #8274

Refactors `DevMode` and `DevSpecification` into a shared type
`DevGroupsSpecification` that allows us to track if `--dev` was
implicitly or explicitly provided.
charliermarsh pushed a commit that referenced this issue Oct 22, 2024
charliermarsh pushed a commit that referenced this issue Oct 22, 2024
Part of #8090

Most of the heavy lifting is done in #8309
charliermarsh pushed a commit that referenced this issue Oct 22, 2024
…8338)

Part of #8090

Most of the heavy lifting is done in #8309

Includes `--only-dev` which appears to be missing as an oversight.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compatibility Compatibility with a specification or another tool enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants