Dependency management improvements #188
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Turns out I've been using the constraint file incorrectly for years. I love learning moments like this! The `constraints.txt` file **is** the lockfile for the project. It contains all the versions that *should* be used *if* they are needed. Is is not the source of **what** should be installed, only what version. So, instead of having N different `requirements*.in` files with N different `requirements*.txt` generated from `pip-compile`; we have a single `constraints.txt` that all of our `requirements*.txt` files reference. Glorious. This solves for the scaling problem that I ran into when, for example, `requirements-doc.txt` had a version conflicct with something in `requirements-test.txt`. Initially, I thought to somehow cross-reference all the `.txt` files with themselves in some horrible chained constraint system. That worked about as well as it sounds. Note, I have to remove the dynamic dependencies here from the `pyproject.toml`. That's fine. Eventually, I'll be moving to `uv` which simplifies most of this juggling. For now, however, I want to understand the underlying machanics before abstracting them away.
Moving toward simplicity of use, the `noxfile.txt` now has a session called `dev`. This session creates, if needed, the `.venv` and installs all development requirements. Additionally, the `update` and `upgrade` sessions have been renamed to `update-deps` and `upgrade-deps` respectively. They now use the new, awesome pattern of a single constraint file for dependencies. Finally, added the `--no-annotations` to the `pip-compile` commands and what a difference that makes! I have yet to understand when and why `pip-compile` will expand `requirements/requirements.txt` to `/home/preocts/project-name/requirements/requirements.txt`. In practice, the results are different depending on the OS being used as well. Since it all lead to noisy diffs without much value, removing it seems ideal!
Okay, this is the ugly part of the path I'm taking. It is now required to install the correct requirement*.txt files for each session. Previously it was a simple `.[test]` reference when installing the package. This too shall pass. Already thinking about pivoting into listing all requirements in the `pyproject.toml`. With dependency groups being supported this should be amazing. It will hindge on `pip-compile` being able to extract from dependency groups. That's the next PR!
Adjust the contributing docs with the new expected nox workflows. Add some leading links at the top of the readme pointing to the secondary files. Reading, strangely, isn't the strongest skill of many.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change is a collection of improvements on how the dependencies of a project are managed. Each commit has a detailed breakdown of its changes.
Added
requestsas a dependency just to have something to demonstrateChanged
constraints.txtfile to rule all version pinninginstallis nowdevupdateis nowupdate-depsupgradeis nowupgrade-deps