Unclear dependency version conflict when installing projects hosted on GitHub with the new resolver

[anjakefala]

Thanks folks for so much intense hard work! I ran into what I think is a bug, and was able to reproduce it in a minimalist environment, and @brainwane encouraged me to share it with you. If it is new expected behaviour, maybe I can help with improving error messaging or adding detail to docs.

1. Operating System
   Ubuntu 18.04.5
2. Python Version
   Python 3.8
3. Pip version (using the new resolver)
   pip 20.3.3

Here is the simplest way to reproduce my problem, along with a theory of what is happening!

I have created 4 sample projects, by copying https://github.com/pypa/sampleproject and changing name and install_requires.

project_A
project_B: dependent on project_A @ git+https://github.com/anjakefala/project_A#egg=project_A
project_C: dependent on project_B @ git+https://github.com/anjakefala/project_B, project_A @ https://github.com/anjakefala/project_A
project_D: dependent on project_B @ git+https://github.com/anjakefala/project_B#egg=project_B

1. Clone and install project_D with pip3 install .
2. Clone and install project_C with pip3 install .

Result:

INFO: pip is looking at multiple versions of project-a to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of project-c to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install project-C and project-c==2.0.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    project-c 2.0.0 depends on project-a 2.0.0 (from git+https://github.com/anjakefala/project_A)
    project-b 2.0.0 depends on project-a 2.0.0 (from git+https://github.com/anjakefala/project_A#egg=project_A)

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

I think the reason this is happening is that project_C does not specify an egg for project_A and project_B does specify an egg. If you turn on the legacy resolver, all of the packages will install, and you will not run into an error.

Other things I ran into is someone has -e in their requirements.txt for a github url, because they read that online somewhere. I believe its version will pinned as the commit hash, as opposed to the version in its setup.py.... I think. is that true?

I did want to mention our psychology behind using github urls. What I am asking pip to do -> "Do not install, if already installed, and upgrade to latest HEAD, if --upgrade is specified. "

I never want to ask pip to "pin to this commit hash specifically, and do not allow other packages to install if the HEAD commit for that same repository happens to be different when they are installed".

But, I could understand that my lab scenario is odd, and I think ultimately I just want to understand is what would be the best way for me to work with the new resolver going forward. Right now, I am working on a personal python index server, where I am hosting wheels, and then pip just grabs the version from the wheel name, and everyone is happy.

Thanks for reading this far!

[anjakefala]

This is very likely related: pypa/pip#9217