Skip to content

Commit

Permalink
Implement --overwrite-pin (#60)
Browse files Browse the repository at this point in the history 
* Implement `--overwrite-pin`

* Update files from markdown-code-runner
  • Loading branch information
@basnijholt
basnijholt authored Dec 8, 2023
1 parent cce86c0 commit a5cd5e0
Show file tree
Hide file tree
Showing 6 changed files with 155 additions and 6 deletions.
41 changes: 38 additions & 3 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -257,7 +257,7 @@ usage: unidep merge [-h] [-o OUTPUT] [-n NAME] [--stdout]
[--selector {sel,comment}] [-d DIRECTORY] [-v]
[--platform {linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64}]
[--depth DEPTH] [--skip-dependency SKIP_DEPENDENCY]
[--ignore-pin IGNORE_PIN]
[--ignore-pin IGNORE_PIN] [--overwrite-pin OVERWRITE_PIN]

Combine multiple (or a single) `requirements.yaml` files into a single Conda
installable `environment.yaml` file. Example usage: `unidep merge --directory
Expand Down Expand Up @@ -297,6 +297,11 @@ options:
Ignore the version pin for a specific package, e.g.,
`--ignore-pin numpy`. This option can be repeated to
ignore multiple packages.
--overwrite-pin OVERWRITE_PIN
Overwrite the version pin for a specific package,
e.g., `--overwrite-pin 'numpy==1.19.2'`. This option
can be repeated to overwrite the pins of multiple
packages.
```
<!-- OUTPUT:END -->
Expand All @@ -319,6 +324,7 @@ usage: unidep install [-h] [-v] [-e] [--skip-local] [--skip-pip]
[--no-dependencies]
[--conda-executable {conda,mamba,micromamba}]
[--dry-run] [--ignore-pin IGNORE_PIN]
[--overwrite-pin OVERWRITE_PIN]
files [files ...]

Automatically install all dependencies from one or more `requirements.yaml`
Expand Down Expand Up @@ -362,6 +368,11 @@ options:
Ignore the version pin for a specific package, e.g.,
`--ignore-pin numpy`. This option can be repeated to
ignore multiple packages.
--overwrite-pin OVERWRITE_PIN
Overwrite the version pin for a specific package,
e.g., `--overwrite-pin 'numpy==1.19.2'`. This option
can be repeated to overwrite the pins of multiple
packages.
```
<!-- OUTPUT:END -->
Expand All @@ -384,6 +395,7 @@ usage: unidep install [-h] [-v] [-e] [--skip-local] [--skip-pip]
[--no-dependencies]
[--conda-executable {conda,mamba,micromamba}]
[--dry-run] [--ignore-pin IGNORE_PIN]
[--overwrite-pin OVERWRITE_PIN]
files [files ...]

Automatically install all dependencies from one or more `requirements.yaml`
Expand Down Expand Up @@ -427,6 +439,11 @@ options:
Ignore the version pin for a specific package, e.g.,
`--ignore-pin numpy`. This option can be repeated to
ignore multiple packages.
--overwrite-pin OVERWRITE_PIN
Overwrite the version pin for a specific package,
e.g., `--overwrite-pin 'numpy==1.19.2'`. This option
can be repeated to overwrite the pins of multiple
packages.
```
<!-- OUTPUT:END -->
Expand All @@ -450,6 +467,7 @@ usage: unidep conda-lock [-h] [--only-global] [--lockfile LOCKFILE]
[--platform {linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64}]
[--depth DEPTH] [--skip-dependency SKIP_DEPENDENCY]
[--ignore-pin IGNORE_PIN]
[--overwrite-pin OVERWRITE_PIN]

Generate a global `conda-lock.yml` file for a collection of
`requirements.yaml` files. Additionally, create individual `conda-lock.yml`
Expand Down Expand Up @@ -490,6 +508,11 @@ options:
Ignore the version pin for a specific package, e.g.,
`--ignore-pin numpy`. This option can be repeated to
ignore multiple packages.
--overwrite-pin OVERWRITE_PIN
Overwrite the version pin for a specific package,
e.g., `--overwrite-pin 'numpy==1.19.2'`. This option
can be repeated to overwrite the pins of multiple
packages.
```
<!-- OUTPUT:END -->
Expand All @@ -510,7 +533,8 @@ See `unidep pip -h` for more information:
usage: unidep pip [-h] [-f FILE] [-v]
[--platform {linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64}]
[--skip-dependency SKIP_DEPENDENCY]
[--ignore-pin IGNORE_PIN] [--separator SEPARATOR]
[--ignore-pin IGNORE_PIN] [--overwrite-pin OVERWRITE_PIN]
[--separator SEPARATOR]
Get the pip requirements for the current platform only. Example usage: `unidep
pip --file folder1 --file folder2/requirements.yaml --seperator ' ' --platform
Expand Down Expand Up @@ -538,6 +562,11 @@ options:
Ignore the version pin for a specific package, e.g.,
`--ignore-pin numpy`. This option can be repeated to
ignore multiple packages.
--overwrite-pin OVERWRITE_PIN
Overwrite the version pin for a specific package,
e.g., `--overwrite-pin 'numpy==1.19.2'`. This option
can be repeated to overwrite the pins of multiple
packages.
--separator SEPARATOR
The separator between the dependencies, by default ` `
```
Expand All @@ -560,7 +589,8 @@ See `unidep conda -h` for more information:
usage: unidep conda [-h] [-f FILE] [-v]
[--platform {linux-64,linux-aarch64,linux-ppc64le,osx-64,osx-arm64,win-64}]
[--skip-dependency SKIP_DEPENDENCY]
[--ignore-pin IGNORE_PIN] [--separator SEPARATOR]
[--ignore-pin IGNORE_PIN] [--overwrite-pin OVERWRITE_PIN]
[--separator SEPARATOR]
Get the conda requirements for the current platform only. Example usage:
`unidep conda --file folder1 --file folder2/requirements.yaml --seperator ' '
Expand Down Expand Up @@ -588,6 +618,11 @@ options:
Ignore the version pin for a specific package, e.g.,
`--ignore-pin numpy`. This option can be repeated to
ignore multiple packages.
--overwrite-pin OVERWRITE_PIN
Overwrite the version pin for a specific package,
e.g., `--overwrite-pin 'numpy==1.19.2'`. This option
can be repeated to overwrite the pins of multiple
packages.
--separator SEPARATOR
The separator between the dependencies, by default ` `
```
Expand Down
45 changes: 45 additions & 0 deletions tests/test_unidep.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1604,3 +1604,48 @@ def test_pin_star_cuda(tmp_path: Path) -> None:
),
],
}


def test_parse_requirements_with_overwrite_pins(tmp_path: Path) -> None:
p = tmp_path / "requirements.yaml"
p.write_text(
textwrap.dedent(
"""\
dependencies:
- foo >1
- conda: bar * cuda*
""",
),
)
requirements = parse_yaml_requirements(
p,
overwrite_pins=["foo=1", "bar * cpu*"],
verbose=False,
)
assert requirements.requirements == {
"foo": [
Meta(
name="foo",
which="conda",
comment=None,
pin="=1",
identifier="17e5d607",
),
Meta(
name="foo",
which="pip",
comment=None,
pin="=1",
identifier="17e5d607",
),
],
"bar": [
Meta(
name="bar",
which="conda",
comment=None,
pin="* cpu*",
identifier="5eb93b8c",
),
],
}
53 changes: 50 additions & 3 deletions unidep/_cli.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,16 @@ def _add_common_args( # noqa: PLR0912
" e.g., `--ignore-pin numpy`. This option can be repeated"
" to ignore multiple packages.",
)
if "overwrite-pin" in options:
sub_parser.add_argument(
"--overwrite-pin",
type=str,
action="append",
default=[],
help="Overwrite the version pin for a specific package,"
" e.g., `--overwrite-pin 'numpy==1.19.2'`. This option can be repeated"
" to overwrite the pins of multiple packages.",
)


def _parse_args() -> argparse.Namespace:
Expand Down Expand Up @@ -237,7 +247,15 @@ def _parse_args() -> argparse.Namespace:
)
_add_common_args(
parser_merge,
{"directory", "verbose", "platform", "depth", "ignore-pin", "skip-dependency"},
{
"directory",
"verbose",
"platform",
"depth",
"ignore-pin",
"skip-dependency",
"overwrite-pin",
},
)

# Subparser for the 'install' command
Expand Down Expand Up @@ -277,6 +295,7 @@ def _parse_args() -> argparse.Namespace:
"no-dependencies",
"ignore-pin",
"skip-dependency",
"overwrite-pin",
"verbose",
},
)
Expand Down Expand Up @@ -315,6 +334,7 @@ def _parse_args() -> argparse.Namespace:
"no-dependencies",
"ignore-pin",
"skip-dependency",
"overwrite-pin",
"verbose",
},
)
Expand Down Expand Up @@ -363,7 +383,15 @@ def _parse_args() -> argparse.Namespace:
)
_add_common_args(
parser_lock,
{"directory", "verbose", "platform", "depth", "ignore-pin", "skip-dependency"},
{
"directory",
"verbose",
"platform",
"depth",
"ignore-pin",
"skip-dependency",
"overwrite-pin",
},
)

# Subparser for the 'pip' and 'conda' command
Expand Down Expand Up @@ -391,7 +419,14 @@ def _parse_args() -> argparse.Namespace:
for sub_parser in [parser_pip, parser_conda]:
_add_common_args(
sub_parser,
{"verbose", "platform", "file", "ignore-pin", "skip-dependency"},
{
"verbose",
"platform",
"file",
"ignore-pin",
"skip-dependency",
"overwrite-pin",
},
)
sub_parser.add_argument(
"--separator",
Expand Down Expand Up @@ -479,6 +514,7 @@ def _install_command(
skip_conda: bool = False,
no_dependencies: bool = False,
ignore_pins: list[str] | None = None,
overwrite_pins: list[str] | None = None,
skip_dependencies: list[str] | None = None,
verbose: bool = False,
) -> None:
Expand All @@ -490,6 +526,7 @@ def _install_command(
requirements = parse_yaml_requirements(
*files,
ignore_pins=ignore_pins,
overwrite_pins=overwrite_pins,
skip_dependencies=skip_dependencies,
verbose=verbose,
)
Expand Down Expand Up @@ -574,6 +611,7 @@ def _install_all_command(
skip_conda: bool = False,
no_dependencies: bool = False,
ignore_pins: list[str] | None = None,
overwrite_pins: list[str] | None = None,
skip_dependencies: list[str] | None = None,
verbose: bool = False,
) -> None: # pragma: no cover
Expand All @@ -595,6 +633,7 @@ def _install_all_command(
skip_conda=skip_conda,
no_dependencies=no_dependencies,
ignore_pins=ignore_pins,
overwrite_pins=overwrite_pins,
skip_dependencies=skip_dependencies,
verbose=verbose,
)
Expand All @@ -611,6 +650,7 @@ def _merge_command(
platforms: list[Platform],
ignore_pins: list[str],
skip_dependencies: list[str],
overwrite_pins: list[str],
verbose: bool,
) -> None: # pragma: no cover
# When using stdout, suppress verbose output
Expand All @@ -627,6 +667,7 @@ def _merge_command(
requirements = parse_yaml_requirements(
*found_files,
ignore_pins=ignore_pins,
overwrite_pins=overwrite_pins,
skip_dependencies=skip_dependencies,
verbose=verbose,
)
Expand Down Expand Up @@ -688,6 +729,7 @@ def main() -> None:
platforms=args.platform,
ignore_pins=args.ignore_pin,
skip_dependencies=args.skip_dependency,
overwrite_pins=args.overwrite_pin,
verbose=args.verbose,
)
elif args.command == "pip": # pragma: no cover
Expand All @@ -698,6 +740,7 @@ def main() -> None:
verbose=args.verbose,
ignore_pins=args.ignore_pin,
skip_dependencies=args.skip_dependency,
overwrite_pins=args.overwrite_pin,
),
)
print(escape_unicode(args.separator).join(pip_dependencies))
Expand All @@ -706,6 +749,7 @@ def main() -> None:
args.file,
ignore_pins=args.ignore_pin,
skip_dependencies=args.skip_dependency,
overwrite_pins=args.overwrite_pin,
verbose=args.verbose,
)
resolved_requirements = resolve_conflicts(requirements.requirements)
Expand All @@ -728,6 +772,7 @@ def main() -> None:
no_dependencies=args.no_dependencies,
ignore_pins=args.ignore_pin,
skip_dependencies=args.skip_dependency,
overwrite_pins=args.overwrite_pin,
verbose=args.verbose,
)
elif args.command == "install-all":
Expand All @@ -744,6 +789,7 @@ def main() -> None:
no_dependencies=args.no_dependencies,
ignore_pins=args.ignore_pin,
skip_dependencies=args.skip_dependency,
overwrite_pins=args.overwrite_pin,
verbose=args.verbose,
)
elif args.command == "conda-lock": # pragma: no cover
Expand All @@ -755,6 +801,7 @@ def main() -> None:
only_global=args.only_global,
ignore_pins=args.ignore_pin,
skip_dependencies=args.skip_dependency,
overwrite_pins=args.overwrite_pin,
check_input_hash=args.check_input_hash,
lockfile=args.lockfile,
)
Expand Down
4 changes: 4 additions & 0 deletions unidep/_conda_lock.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ def _conda_lock_global(
check_input_hash: bool,
ignore_pins: list[str],
skip_dependencies: list[str],
overwrite_pins: list[str],
lockfile: str,
) -> Path:
"""Generate a conda-lock file for the global dependencies."""
Expand All @@ -103,6 +104,7 @@ def _conda_lock_global(
selector="comment",
platforms=platform,
ignore_pins=ignore_pins,
overwrite_pins=overwrite_pins,
skip_dependencies=skip_dependencies,
verbose=verbose,
)
Expand Down Expand Up @@ -443,6 +445,7 @@ def conda_lock_command(
check_input_hash: bool,
ignore_pins: list[str],
skip_dependencies: list[str],
overwrite_pins: list[str],
lockfile: str = "conda-lock.yml",
) -> None:
"""Generate a conda-lock file a collection of requirements.yaml files."""
Expand All @@ -453,6 +456,7 @@ def conda_lock_command(
verbose=verbose,
check_input_hash=check_input_hash,
ignore_pins=ignore_pins,
overwrite_pins=overwrite_pins,
skip_dependencies=skip_dependencies,
lockfile=lockfile,
)
Expand Down
Loading

0 comments on commit a5cd5e0

Please sign in to comment.