`python_requires="<4"` creates issues with dependency solvers

[engnadeau]

• python_requires="<4" is defined in https://github.com/nedbat/coveragepy/blob/aa6cb95e366f479cddd8cd30aee940393ead6296/setup.py#L122
• This requirement causes problems with dependency solvers, such as poetry:

>>> poetry add --allow-prereleases --dev coverage          
Using version ^5.2.1 for coverage

Updating dependencies
Resolving dependencies... (0.0s)

[SolverProblemError]
The current project's Python requirement (>=3.6) is not compatible with some of the required packages Python requirement:
  - coverage requires Python >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <4

[nedbat]

This seems like something that should be reported to poetry, no? Why can't they understand "<4"? Would "<4.0" work?

[engnadeau]

Bit of both. Poetry has the functionality but it depends on how people configure their dependency limits (e.g., when people come over from another packaging tool).

• If someone were to use python = "^3.6.1" in their config, then that expands to >=3.6.1, <4.0.0
• If someone were to use python = ">=3.6.1" in their config, then the install would fail due to the <4.0.0 clause

Yes, people should configure their packaging tool correctly, but since python4 doesn't exist yet (and it's not on the roadmap yet?), why have that additional clause in setup.py?

[nedbat]

I guess it's fine to remove the <4 constraint, but I'm trying to understand:

If someone were to use python = ">=3.6.1" in their config, then the install would fail due to the <4.0.0 clause

Why would ">=3.6.1" and "<4.0.0" conflict?

[engnadeau]

Dependency resolution in this context takes things literally. >=3.6.1 can include versions >=4. Since coveragepy specifies <4 (i.e., >=4 is NOT allowed), this creates a conflict.

[nedbat]

If you use python = "^3.6", it will work. I don't understand why poetry can't find the intersection of ">3.6" and "<4", but at least it understands "^3.6" and "<4".

[nedbat]

When I try this with poetry, it says:

$ poetry add coverage
Using version ^5.3 for coverage

Updating dependencies
Resolving dependencies... (0.0s)

  SolverProblemError

  The current project's Python requirement (>=3.8) is not compatible with some of the required packages Python requirement:
    - coverage requires Python >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <4, so it will not be satisfied for Python >=4

  Because no versions of coverage match >5.3,<6.0
   and coverage (5.3) requires Python >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <4, coverage is forbidden.
  So, because repro depends on coverage (^5.3), version solving failed.

  at /usr/local/virtualenvs/tmp-ea702c341d4f3fc4/lib/python3.8/site-packages/poetry/puzzle/solver.py:241 in _solve
      237│             packages = result.packages
      238│         except OverrideNeeded as e:
      239│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
      240│         except SolveFailure as e:
    → 241│             raise SolverProblemError(e)
      242│
      243│         results = dict(
      244│             depth_first_search(
      245│                 PackageNode(self._package, packages), aggregate_package_nodes

  • Check your dependencies Python requirement: The Python requirement can be specified via the `python` or `markers` properties

    For coverage, a possible solution would be to set the `python` property to ">=3.8,<4"

    https://python-poetry.org/docs/dependency-specification/#python-restricted-dependencies,
    https://python-poetry.org/docs/dependency-specification/#using-environment-markers

Did you get this message? Did you consider adding "<4" to your python property? I'm trying to understand how people get into this state.

[nedbat]

Poetry seems set on continuing to behave this way: python-poetry/poetry#1930 (comment)

[nedbat]

@nnadeau can you help me understand how you ended up with ">=3.6" in your pyproject.yaml? Poetry initializes it with "^3.6", which works.

[engnadeau]

@nedbat it was an artifact of transitioning from a pipenv-based project to a poetry project. Pipenv doesn't appear to be as strict with dependency resolution