Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Supply the extra pip args in the resolver. #6006

Merged
merged 5 commits into from
Jun 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions docs/locking.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Locking Dependencies

Locking dependencies in Pipenv ensures that your project's dependencies are consistent across different environments by creating a `Pipfile.lock` file. This file contains the exact versions of all dependencies, making your builds deterministic and reproducible.

## Creating a Lock File

Pipenv will automatically generate a `Pipfile.lock` file when you install dependencies using the `pipenv install` command. This file contains the specific versions of all installed packages and their dependencies.

```bash
pipenv install requests
```

To completely update a `Pipfile.lock` file based on the currently available packages and your `Pipfile` specifiers, run the following command:

pipenv lock


## Installing from a Lock File

When you have a `Pipfile.lock` file in your project, you can install the exact dependencies specified in the lock file with:

pipenv install

## Updating Lock Files

When you need to update your dependencies, you can update a subset of lock file with:

pipenv upgrade mypackage==1.2.3

This command updates all dependencies to their latest compatible versions and regenerates the `Pipfile.lock`.

Should you want `pipenv` to also install the upgraded packages, you can run the following command:

pipenv update mypackage==1.2.3

## Viewing Locked Dependencies

You can view the currently locked dependencies and their versions with:

pipenv graph

This command outputs a dependency graph, showing all locked packages and their dependencies.

## Best Practices

- **Commit Your `Pipfile.lock`:** Always commit your `Pipfile.lock` file to version control to ensure that your team members and CI/CD pipelines use the same dependencies.
- **Regularly Update Dependencies:** Periodically update your dependencies to include the latest patches and improvements while ensuring compatibility.
- **Review Dependency Changes:** When updating dependencies, review the changes in the `Pipfile.lock` to understand the impact on your project.

## Troubleshooting

### Common Issues

- **Dependency Conflicts:** If you encounter dependency conflicts, you may need to adjust your `Pipfile` or manually resolve the conflicts before locking again. Run with the `--verbose` flag to get more information about the conflict.
- **Installation Errors:** Ensure that your `Pipfile.lock` is not corrupted and that you have the necessary permissions to install the dependencies. Check for common system dependencies when building sdist packages.

### Helpful Commands

- **Check for Dependency Issues:**

```bash
pipenv check
```

This command checks for security vulnerabilities and other issues in your dependencies.

- **Clear Caches:**

```bash
pipenv --clear
```

This command clears Pipenv’s caches, which can resolve some installation issues.

## Supplying additional arguments to the pipenv resolver

You can supply additional arguments to the pipenv resolver by supplying `--extra-pip-arg` to the `install` command. For example, `pipenv install --extra-pip-args="--platform=win_amd64 --target /path/to/my/target`.

```bash

## Conclusion

Locking in Pipenv is a powerful feature that ensures your Python project’s dependencies are consistent, secure, and reproducible. By understanding and utilizing Pipenv’s locking mechanism, you can avoid many common dependency management issues and ensure your projects run smoothly across different environments.

```note
Locking ensures that all dependencies are resolved and installed in a consistent manner. This helps avoid issues where different versions of dependencies might introduce bugs or inconsistencies in your project.
1 change: 1 addition & 0 deletions news/6006.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Supply any ``--extra-pip-args`` also in the resolver steps.
1 change: 1 addition & 0 deletions pipenv/cli/command.py
Original file line number Diff line number Diff line change
Expand Up @@ -258,6 +258,7 @@ def upgrade(state, **kwargs):
dev=state.installstate.dev,
system=state.system,
lock_only=state.installstate.lock_only,
extra_pip_args=state.installstate.extra_pip_args,
)


Expand Down
2 changes: 2 additions & 0 deletions pipenv/routines/lock.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ def do_lock(
write=True,
pypi_mirror=None,
categories=None,
extra_pip_args=None,
):
"""Executes the freeze functionality."""
if not pre:
Expand Down Expand Up @@ -74,6 +75,7 @@ def do_lock(
pipfile=packages,
lockfile=lockfile,
old_lock_data=old_lock_data,
extra_pip_args=extra_pip_args,
)

# Overwrite any category packages with default packages.
Expand Down
8 changes: 8 additions & 0 deletions pipenv/routines/update.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import json
import os
import sys
from collections import defaultdict

Expand Down Expand Up @@ -62,6 +64,7 @@ def do_update(
pre=pre,
pypi_mirror=pypi_mirror,
write=not outdated,
extra_pip_args=extra_pip_args,
)
else:
upgrade(
Expand All @@ -75,6 +78,7 @@ def do_update(
index_url=index_url,
dev=dev,
lock_only=lock_only,
extra_pip_args=extra_pip_args,
)

if outdated:
Expand Down Expand Up @@ -110,6 +114,7 @@ def upgrade(
categories=None,
dev=False,
lock_only=False,
extra_pip_args=None,
matteius marked this conversation as resolved.
Show resolved Hide resolved
):
lockfile = project.lockfile()
if not pre:
Expand All @@ -123,6 +128,9 @@ def upgrade(
if index_url:
index_name = add_index_to_pipfile(project, index_url)

if extra_pip_args:
os.environ["PIPENV_EXTRA_PIP_ARGS"] = json.dumps(extra_pip_args)

package_args = list(packages) + [f"-e {pkg}" for pkg in editable_packages]

requested_install_reqs = defaultdict(dict)
Expand Down
11 changes: 10 additions & 1 deletion pipenv/utils/resolver.py
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,10 @@ def prepare_pip_args(self, use_pep517=None, build_isolation=True):
if self.pre:
pip_args.append("--pre")
pip_args.extend(["--cache-dir", self.project.s.PIPENV_CACHE_DIR])
extra_pip_args = os.environ.get("PIPENV_EXTRA_PIP_ARGS")
if extra_pip_args:
extra_pip_args = json.loads(extra_pip_args)
pip_args.extend(extra_pip_args)
return pip_args

@property # cached_property breaks authenticated private indexes
Expand Down Expand Up @@ -748,6 +752,7 @@ def venv_resolve_deps(
pipfile=None,
lockfile=None,
old_lock_data=None,
extra_pip_args=None,
):
"""
Resolve dependencies for a pipenv project, acts as a portal to the target environment.
Expand Down Expand Up @@ -799,7 +804,11 @@ def venv_resolve_deps(
os.environ["PIPENV_SITE_DIR"] = pipenv_site_dir
else:
os.environ.pop("PIPENV_SITE_DIR", None)
with console.status("Locking...", spinner=project.s.PIPENV_SPINNER) as st:
if extra_pip_args:
os.environ["PIPENV_EXTRA_PIP_ARGS"] = json.dumps(extra_pip_args)
with console.status(
f"Locking {category}...", spinner=project.s.PIPENV_SPINNER
) as st:
# This conversion is somewhat slow on local and file-type requirements since
# we now download those requirements / make temporary folders to perform
# dependency resolution on them, so we are including this step inside the
Expand Down
Loading