pip-compile fails to compute dependencies in scenarios when other tooling can

[takeda]

If you for example try to compute dependencies for pyrax==1.9.8 it will fail.

It is mentioned here: https://www.reddit.com/r/Python/comments/8lapa5/comparison_of_pipenv_vs_piptools/dzea4xr

Steps to replicate

1. create requirements.in containing pyrax==1.9.8
2. run pip-compile and see the error

Expected result

figure out the dependencies

Actual result

Could not find a version that matches pbr!=2.1.0,<2.0,>=0.11,>=1.6,>=2.0.0
Tried: 0.5.2.5.g5b3e942, 0.5.0, 0.5.1, 0.5.2, 0.5.4, 0.5.5, 0.5.6, 0.5.7, 0.5.8, 0.5.10, 0.5.11, 0.5.12, 0.5.13, 0.5.14, 0.5.15, 0.5.16, 0.5.17, 0.5.18, 0.5.19, 0.5.20, 0.5.21, 0.5.22, 0.5.23, 0.6, 0.7.0, 0.8.0, 0.8.1, 0.8.2, 0.9.0, 0.9.0, 0.10.0, 0.10.0, 0.10.1, 0.10.1, 0.10.2, 0.10.2, 0.10.3, 0.10.3, 0.10.4, 0.10.4, 0.10.5, 0.10.5, 0.10.6, 0.10.6, 0.10.7, 0.10.7, 0.10.8, 0.10.8, 0.11.0, 0.11.0, 0.11.1, 0.11.1, 1.0.0, 1.0.0, 1.0.1, 1.0.1, 1.1.0, 1.1.0, 1.1.1, 1.1.1, 1.2.0, 1.2.0, 1.3.0, 1.3.0, 1.4.0, 1.4.0, 1.5.0, 1.5.0, 1.6.0, 1.6.0, 1.7.0, 1.7.0, 1.8.0, 1.8.0, 1.8.1, 1.8.1, 1.9.0, 1.9.0, 1.9.1, 1.9.1, 1.10.0, 1.10.0, 2.0.0, 2.0.0, 2.1.0, 2.1.0, 3.0.0, 3.0.0, 3.0.1, 3.0.1, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 4.0.0, 4.0.0, 4.0.1, 4.0.1, 4.0.2, 4.0.2, 4.0.3, 4.0.3, 4.0.4, 4.0.4, 4.1.0, 4.1.0, 4.1.1, 4.1.1, 4.2.0, 4.2.0, 4.3.0, 4.3.0, 5.0.0, 5.0.0
There are incompatible versions in the resolved dependencies.

...

[atugushev]

That's because pip-tools doesn't have a proper dependency resolver.

[AndydeCleyre]

See:

• pip #988
• update on #988
• pip #7317
• pip #6536
• zazo
• resolvelib

[atugushev]

@AndydeCleyre thanks for the links collection!

[NeolithEra]

Hi, maybe you can try a dependency diagnosis website http://www.watchman-pypi.com/dependency_analysis, which can help diagnoze dependency conflict problems and automatically visualize your dependency tree and those of your downstream projects.

Best,
Neolith

[AndydeCleyre]

From a new comment from brainwane on pip 988:

We aim to have pip's resolver feature prepared for release in pip 20.2 in July. (Per the quarterly release cadence for pip, unforeseen difficulties may delay till 20.3 in the next quarter.)

[atugushev]

#1539 finally adds support for 2020 dependency resolver. Try pip-compile --resolver backtracking.

Expand the output

$ cat requitements.in
pyrax==1.9.8

$ pip-compile --resolver 2020
#
# This file is autogenerated by pip-compile with python 3.8
# To update, run:
#
#    pip-compile --resolver=2020
#
argparse==1.4.0
    # via python-novaclient
babel==2.9.1
    # via
    #   oslo-i18n
    #   python-novaclient
certifi==2021.10.8
    # via requests
charset-normalizer==2.0.7
    # via requests
debtcollector==1.11.0
    # via
    #   oslo-config
    #   oslo-utils
    #   python-keystoneclient
idna==3.3
    # via requests
importlib-metadata==4.8.2
    # via keyring
ip-associations-python-novaclient-ext==0.2
    # via rackspace-novaclient
iso8601==1.0.0
    # via
    #   keystoneauth1
    #   oslo-utils
    #   python-novaclient
keyring==23.2.1
    # via pyrax
keystoneauth1==2.18.0
    # via python-keystoneclient
mock==4.0.3
    # via pyrax
monotonic==1.6
    # via oslo-utils
msgpack-python==0.5.6
    # via oslo-serialization
netaddr==0.8.0
    # via
    #   oslo-config
    #   oslo-utils
netifaces==0.11.0
    # via oslo-utils
os-diskconfig-python-novaclient-ext==0.1.3
    # via rackspace-novaclient
os-networksv2-python-novaclient-ext==0.26
    # via rackspace-novaclient
os-virtual-interfacesv2-python-novaclient-ext==0.20
    # via rackspace-novaclient
oslo-config==4.12.0
    # via python-keystoneclient
oslo-i18n==3.12.0
    # via
    #   oslo-config
    #   oslo-utils
    #   python-keystoneclient
    #   python-novaclient
oslo-serialization==2.16.1
    # via
    #   python-keystoneclient
    #   python-novaclient
oslo-utils==3.22.3
    # via
    #   oslo-serialization
    #   python-keystoneclient
    #   python-novaclient
pbr==1.10.0
    # via
    #   debtcollector
    #   keystoneauth1
    #   oslo-i18n
    #   oslo-serialization
    #   oslo-utils
    #   positional
    #   python-keystoneclient
    #   python-novaclient
    #   stevedore
positional==1.2.1
    # via
    #   keystoneauth1
    #   python-keystoneclient
prettytable==0.7.2
    # via python-novaclient
pyparsing==3.0.6
    # via oslo-utils
pyrax==1.9.8
    # via -r requirements.in
python-keystoneclient==3.10.0
    # via python-novaclient
python-novaclient==2.27.0
    # via
    #   ip-associations-python-novaclient-ext
    #   os-diskconfig-python-novaclient-ext
    #   os-networksv2-python-novaclient-ext
    #   os-virtual-interfacesv2-python-novaclient-ext
    #   pyrax
    #   rackspace-auth-openstack
    #   rackspace-novaclient
    #   rax-default-network-flags-python-novaclient-ext
    #   rax-scheduled-images-python-novaclient-ext
pytz==2021.3
    # via
    #   babel
    #   oslo-serialization
    #   oslo-utils
pyyaml==6.0
    # via oslo-config
rackspace-auth-openstack==1.3
    # via rackspace-novaclient
rackspace-novaclient==2.1
    # via pyrax
rax-default-network-flags-python-novaclient-ext==0.4.0
    # via rackspace-novaclient
rax-scheduled-images-python-novaclient-ext==0.3.1
    # via rackspace-novaclient
requests==2.26.0
    # via
    #   keystoneauth1
    #   pyrax
    #   python-keystoneclient
    #   python-novaclient
rfc3986==1.5.0
    # via oslo-config
simplejson==3.17.6
    # via python-novaclient
six==1.16.0
    # via
    #   debtcollector
    #   keystoneauth1
    #   oslo-config
    #   oslo-i18n
    #   oslo-serialization
    #   oslo-utils
    #   pyrax
    #   python-keystoneclient
    #   python-novaclient
    #   stevedore
stevedore==1.20.1
    # via
    #   keystoneauth1
    #   oslo-config
    #   python-keystoneclient
urllib3==1.26.7
    # via requests
wrapt==1.13.3
    # via
    #   debtcollector
    #   positional
zipp==3.6.0
    # via importlib-metadata

[atugushev]

This has been fixed in #1539 with the backtracking resolver, try pip-compile --resolver backtracking. The resolver is released as part of pip-tools v6.8.0. Please let us know if it doesn't resolve your issue. Thanks!