After pip 10 upgrade on pyenv "ImportError: cannot import name 'main'"

[KleinerNull]

Maintainer note: Anyone that still gets this issue please see #5599.

---

• Pip version: 10.0
• Python version: 3.6.2
• Operating system: Ubuntu 16.04

Description:

After upgrading pip from 9.03 to 10.0 via pip install pip --user --upgrade in a pyenv environment pip refueses to start and raise this instead:

Traceback (most recent call last):
  File "/home/kleinernull/.pyenv/versions/3.6.2/bin/pip", line 7, in <module>
    from pip import main
ImportError: cannot import name 'main'
Traceback (most recent call last):
  File "/home/kleinernull/.pyenv/versions/3.6.2/bin/pip", line 7, in <module>
    from pip import main
ImportError: cannot import name 'main'

The content of all three different pip files is the same:

~ ⟩ cat .pyenv/versions/3.6.2/bin/pip                                                                            ~
#!/home/kleinernull/.pyenv/versions/3.6.2/bin/python3.6

# -*- coding: utf-8 -*-
import re
import sys

from pip._internal import main as _main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(_main())

~ ⟩ cat .pyenv/versions/3.6.2/bin/pip3                                                                           ~
#!/home/kleinernull/.pyenv/versions/3.6.2/bin/python3.6


# -*- coding: utf-8 -*-
import re
import sys

from pip import main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

~ ⟩ cat .pyenv/versions/3.6.2/bin/pip3.6                                                                         ~
#!/home/kleinernull/.pyenv/versions/3.6.2/bin/python3.6

# -*- coding: utf-8 -*-
import re
import sys

from pip import main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

The same happend with my 3.6.1 environment too.

Temporaly fix

According to the code of the master branch the import should be that:

#!/home/kleinernull/.pyenv/versions/3.6.2/bin/python3.6

# -*- coding: utf-8 -*-
import re
import sys

from pip._internal import main as _main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(_main())

And this resolves the issue. I have no clue if this has something to do with the upgrade itself or with pyenv as environment.

[pradyunsg]

Hey @KleinerNull!

I have no clue if this has something to do with the upgrade itself or with pyenv as environment.

I think that this is an environment issue. I suggest you open an issue over at pyenv as I think the folks there would be in a better position to comment on this/fix it.

[KleinerNull]

@pradyunsg

I opened a issue at the pyenv repo.

[JohnVonNeumann]

We are suffering from this one as well, it's broken our pipeline. Operation being run:

 11 #upgrade pip and install uwsgi
 12 pip install --user --upgrade pip
 13 pip install uwsgi

Ubuntu 16.04
Python3.6

[HayaoSuzuki]

We are encountering the same trouble.

// in host
$ docker pull ubuntu:xenial
$ docker run --name pip-test --rm -it ubuntu:xenial bash

// in container
# apt update
# apt install -y python-dev python-pip
# pip install --upgrade pip
Collecting pip
  Downloading pip-10.0.0-py2.py3-none-any.whl (1.3MB)
    100% |################################| 1.3MB 865kB/s 
Installing collected packages: pip
  Found existing installation: pip 8.1.1
    Not uninstalling pip at /usr/lib/python2.7/dist-packages, outside environment /usr
Successfully installed pip-10.0.0
# pip install requests
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    from pip import main
ImportError: cannot import name main

[maingoh]

Same here .. Exact same output as @HayaoSuzuki and we don't use pyenv

[pfmoore]

If you're getting this outside of pyenv I suspect it's more likely to be related to #5221

[PhML]

Just to mention that pip install --user --upgrade pip on Ubuntu 16.04 also breaks.

Interestingly easy installing pip installs 10.0.0 but does not exhibit the issue

// in host
$ docker pull ubuntu:xenial
$ docker run --name pip-test --rm -it ubuntu:xenial bash

// in container
# apt update
# apt install -y python-setuptools 
# easy_install pip
Installed /usr/local/lib/python2.7/dist-packages/pip-10.0.0-py2.7.egg
Processing dependencies for pip
Finished processing dependencies for pip

# pip install requests
Collecting requests
 Downloading requests-2.18.4-py2.py3-none-any.whl (88kB)
   100% |################################| 92kB 2.9MB/s 
Collecting certifi>=2017.4.17 (from requests)
 Downloading certifi-2018.1.18-py2.py3-none-any.whl (151kB)
   100% |################################| 153kB 4.4MB/s 
Collecting chardet<3.1.0,>=3.0.2 (from requests)
 Downloading chardet-3.0.4-py2.py3-none-any.whl (133kB)
   100% |################################| 143kB 4.5MB/s 
Collecting idna<2.7,>=2.5 (from requests)
 Downloading idna-2.6-py2.py3-none-any.whl (56kB)
   100% |################################| 61kB 7.4MB/s 
Collecting urllib3<1.23,>=1.21.1 (from requests)
 Downloading urllib3-1.22-py2.py3-none-any.whl (132kB)
   100% |################################| 133kB 4.4MB/s 
Installing collected packages: certifi, chardet, idna, urllib3, requests
Successfully installed certifi-2018.1.18 chardet-3.0.4 idna-2.6 requests-2.18.4 urllib3-1.22

[RonnyPfannschmidt]

which is not surprising, since that pip is egg-installed in a different place thus no longer affecting the global pip - it also means you now have a very interesting workingset for python

[davidjlloyd]

While I appreciate that the underlying issue reported here and in #5221 is the environment, the cause is primarily that the import from pip import main was broken as the package pip.main was moved to pip._internal.main. It would be trivial to add a link from pip.main to pip._internal.main to fix this (whereas fixing the environment is a lot of work across many locations for many people). Is there a good reason to not do this?

[KleinerNull]

@davidjlloyd

This comment gives some informations about not to do this, but I am not entirely sure if this is also important for the pip script itself. However, it solved the problem for me.

[pfmoore]

@davidjlloyd Because from pip import main was never supported, basically. And while it's easy to say "yes, but it's a simple API and it just worked", it really didn't - we've had multiple bug reports from people expecting certain behaviours from it, that we simply don't cater for (running pip.main in multiple threads, expecting pip.main to not change the configuration of the logging system, ...)

Rather than keep explaining to people that they shouldn't do this, and continually dealing with the fact that people are assuming "if I can find a function to call, it's supported" we moved everything to the _internal namespace to make it abundantly clear that you're not supposed to call it.

The bulk of the complaints have come from people using pip.main - which is ironic, as it's so easy to call pip in the supported command line way via subprocess. So of all the possible breakages, this is the easiest to fix - and yet people still haven't fixed it, even after months of warnings. (Although to be fair to the Linux distributions, they don't support people upgrading their system packages using pip, so reports like #5221 of cases where distro-supplied scripts break is fundamentally user error, not failure of the distros to address the pip 10 changes - something I'm sure they are handling perfectly well).