Filter pip environment markers

[JoergRittinger]

Add filter for pip environment markers

Changelog-friendly one-liner: Bugfix: Handling of pip environment markers

Contributor checklist

• Provided the tests for the changes
• Requested (or received) a review from another contributor
• Gave a clear one-line description in the PR (that the maintainers can add to CHANGELOG.md afterwards).

[vphilippon]

Thanks for tackling this one!

Unfortunately, that implementation breaks the current behavior of keeping the markers in the requirements.txt which allows to make simple cases compatible cross-environment. See #635 (comment) and the replies afterward.

What could be done would be to make the filtering inside the Resolver, maybe in check_constraints or somewhere that applies only to the top-level constraints, so that it only excludes them from the dependency resolving part, which is were the issue currently comes from.

While I was originally discussing the removal of the markers in the requirements.txt in the comment I linked, I'm now inclined to keep them, because:

• It keeps the simple cases of (ex: functools32 and enum34) working cross-environment
• It keeps a form of information in the requirements.txt that will indicate that there was originally a marker, and that will help with offering support if someone puts a maker on a package with dependencies and can't figure out why some sub-dependencies aren't included in the requirements.txt.

[JoergRittinger]

Thx @vphilippon for you feedback. I tried an implementation which puts the package with the environment marker into the requirements.txt but excludes it from repository lookups and dependency searches. But it will get inconsistent anyway.

If I have the requirments.in with:
unknown_python_3_package; python_version < '3'
I will end up with the requirements.txt in python 2:
unknown_python_3_package==1.0 ; python_version < '3'
and with the requirements.txt in python 3:
unknown_python_3_package ; python_version < '3'

Note that I can find a version on my Repository server for python 2, but for python 3 it is not available and therefore can't be pinned.

If you want I can push my changes for further discussion.

[vphilippon]

@JoergRittinger Thanks for the detailed example.

Your original idea seems to be the best one in the end.
Plus, it will keep the environment markers in the requirements.txt when the marker evaluates as true, which should cover some easy cross-environment cases, as long as the pip-compile is done on the python version that includes the additional packages (FYI @barrywhart for your usage).

[vphilippon]

Looks Good 👍

[tuukkamustonen]

I believe this got in into 2.0.0? It's missing from the changelog, though?

[vphilippon]

You're right, that's what happens with late night releases. Thanks for pointing it out
Edit: Added to changelog and releases notes. That was definitely an important one.