_NamespacePath object has no attribute sort

[jaraco]

• Pip version: 9.0.1
• Python version: 3.6
• Operating System: macOS

It seems that pip by way of vendoring Setuptools is triggering the issue reported in pypa/setuptools#885 (and may likely be implicated in the ongoing reports of users experiencing the issue). Here's a traceback that occurs when pip freeze is invoked by tox during a test run after completing the develop-inst-no-deps step.

Traceback (most recent call last):
  File "/Users/jaraco/Dropbox/code/yg/support/yg.piistore/.tox/python/bin/pip", line 7, in <module>
    from pip import main
  File "/Users/jaraco/Dropbox/code/yg/support/yg.piistore/.tox/python/lib/python3.6/site-packages/pip/__init__.py", line 26, in <module>
    from pip.utils import get_installed_distributions, get_prog
  File "/Users/jaraco/Dropbox/code/yg/support/yg.piistore/.tox/python/lib/python3.6/site-packages/pip/utils/__init__.py", line 27, in <module>
    from pip._vendor import pkg_resources
  File "/Users/jaraco/Dropbox/code/yg/support/yg.piistore/.tox/python/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3018, in <module>
    @_call_aside
  File "/Users/jaraco/Dropbox/code/yg/support/yg.piistore/.tox/python/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3004, in _call_aside
    f(*args, **kwargs)
  File "/Users/jaraco/Dropbox/code/yg/support/yg.piistore/.tox/python/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3046, in _initialize_master_working_set
    dist.activate(replace=False)
  File "/Users/jaraco/Dropbox/code/yg/support/yg.piistore/.tox/python/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2578, in activate
    declare_namespace(pkg)
  File "/Users/jaraco/Dropbox/code/yg/support/yg.piistore/.tox/python/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2152, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/Users/jaraco/Dropbox/code/yg/support/yg.piistore/.tox/python/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2092, in _handle_ns
    _rebuild_mod_path(path, packageName, module)
  File "/Users/jaraco/Dropbox/code/yg/support/yg.piistore/.tox/python/lib/python3.6/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2121, in _rebuild_mod_path
    orig_path.sort(key=position_in_sys_path)
AttributeError: '_NamespacePath' object has no attribute 'sort'

I imagine the only fix is to re-vendor setuptools with 31.0.1 or later.

[jaraco]

I noticed there was a bug in my setup.py script. The package name had a latent 'skeleton' for the name instead of the proper project name. After correcting that, the issue went away, likely because the proper project name yg.piistore appears in a different sort order than skeleton. Still, I suspect this issue will exist for some users depending on the order and names of packages and their dependencies.

[jaraco]

Perhaps this issue is only triggered if the package being installed is a non-namespace package but it depends on namespace packages.

[arthur-tacca]

I got the same error ('_NamespacePath' object has no attribute 'sort'). I hope the following details of my setup are of some use.

• Pip version: 9.0.1
• Python version: 3.5
• Setuptools version: 32.3.1
• OS: Ubuntu 16.04

Three packages are in play (although the first probably isn't important). I have anonymised the package names.

• acme.foo: This is in a directory on my hard disk, so these are all valid paths:
  • /path/to/acmefoo/ is the project root
  • /path/to/acmefoo/acme/foo contains the source code for the package
  • /path/to/acmefoo/acme/__init__.py contains (only) the text: __import__('pkg_resources').declare_namespace(__name__)
  • /path/to/acmefoo/setup.py includes packages=["acme", "acme.foo"], namespace_packages=["acme"] (and of course uses setuptools, not distutils).
• acme.bar: This is similar, plus its setup.py contains entrypoints={'console_scripts': ['ac_bar = acme.bar.main:main']} (although removing this is not necessary to reproduce).
• acme.baz: This is similar except that it has no setup.py.

I installed acme.foo and acme.bar directly by just running e.g. pip install /path/to/acmefoo/. I then change the current working directory to acme.baz, and running pip freeze or ac_bar gives the error.

Some extra points:

• Deleting /path/to/acmebaz/acme/__init__.py (i.e. the package that was NOT pip installed) stops the error from occurring.
• Going through this whole procedure with setuptools==28.0.0 does NOT reproduce the error.
• If I install acme.foo and acme.bar with setuptools==32.3.1, then downgrade to setuptools==28.0.0, then run pip freeze in /path/to/acmebaz, I DO get the error. So what is important is the version of setuptools used for pip installation, not the version that is currently active.

[jaraco]

If I install, then downgrade setuptools, then run pip freeze, I do get the error.

This behavior almost certainly implicates the changes in Setuptools v31 for pypa/setuptools#250. In particular, pypa/setuptools@7e25fd9 updated the way that nspkg-pth files are generated - this is where the _NamespacePath objects are created.

But it was only in Setuptools 31.0.1 where the sort error was addressed in pkg_resources. However, as you can see in my traceback, the error is occurring in the vendored version of pkg_resources, which doesn't yet have that fix.

What's interesting and I don't yet understand is why deleting the namespace package declaration from acme (for baz) stops the error. I'm unsure what's happening there, but it's probably worthy of some more investigation.

[RitamDey]

Python Version Python 3.6.0b2
Pip Version 9.0.1
Operating System Ubuntu 16.10 (4.9.0-1.1-liquorix-amd64 #1 ZEN SMP PREEMPT liquorix 4.9-3 (2017-01-07) x86_64 x86_64 x86_64 GNU/Linux)

  File "/usr/lib/python3.6/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/lib/python3.6/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/usr/lib/python3.6/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/usr/local/lib/python3.6/dist-packages/pip/__init__.py", line 26, in <module>
    from pip.utils import get_installed_distributions, get_prog
  File "/usr/local/lib/python3.6/dist-packages/pip/utils/__init__.py", line 27, in <module>
    from pip._vendor import pkg_resources
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 3018, in <module>
    @_call_aside
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 3004, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 3046, in _initialize_master_working_set
    dist.activate(replace=False)
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2578, in activate
    declare_namespace(pkg)
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2152, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2092, in _handle_ns
    _rebuild_mod_path(path, packageName, module)
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2121, in _rebuild_mod_path
    orig_path.sort(key=position_in_sys_path)
AttributeError: '_NamespacePath' object has no attribute 'sort'
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
    from apport.report import Report
  File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in <module>
    import apport.fileutils
  File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in <module>
    from apport.packaging_impl import impl as packaging
  File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 23, in <module>
    import apt
  File "/usr/lib/python3/dist-packages/apt/__init__.py", line 23, in <module>
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'

Original exception was:
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/lib/python3.6/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/usr/lib/python3.6/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/usr/local/lib/python3.6/dist-packages/pip/__init__.py", line 26, in <module>
    from pip.utils import get_installed_distributions, get_prog
  File "/usr/local/lib/python3.6/dist-packages/pip/utils/__init__.py", line 27, in <module>
    from pip._vendor import pkg_resources
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 3018, in <module>
    @_call_aside
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 3004, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 3046, in _initialize_master_working_set
    dist.activate(replace=False)
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2578, in activate
    declare_namespace(pkg)
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2152, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2092, in _handle_ns
    _rebuild_mod_path(path, packageName, module)
  File "/usr/local/lib/python3.6/dist-packages/pip/_vendor/pkg_resources/__init__.py", line 2121, in _rebuild_mod_path
    orig_path.sort(key=position_in_sys_path)
AttributeError: '_NamespacePath' object has no attribute 'sort'

I am also getting this error. I tried to downgrading to setuptools v28 but it didn't solved. Also this error doesn't pops up in Python 3.5.2. But now I noticed that pip doesn't have this error in Heroku. I have an Django app running on Python 3.6 inside Heroku and it never has any troubles

[arthur-tacca]

For me the solution was to abandon namespace packages; acme/foo is now acme_foo. There seem to be a few corner cases buried deep in the Python infrastructure (lack of Cython / extension module support is another) and this was the final straw.

[jaraco]

This issue is implicated in these failing tests. Although I had surmised above that the issue might be isolated to non-namespace-packages requiring namespace packages, this example is of a namespace package (pmxbot.rss) requiring another namespace package in the same namespace (pmxbot).

[jaraco]

The issue with those failing tests was corrected by updating the other project to declare its namespace package again.

[kelein]

@jaraco I met the same question, and I don't know how to fix it ? And my pip does not work for installing everything. Help me ! 🆘

Python 3.5.2
Pip 9.0.1
Setuptools 34.2.0

>> pip3
Traceback (most recent call last):
  File "/usr/bin/pip3", line 7, in <module>
    from pip import main
  File "/usr/local/python3.5.2/lib/python3.5/site-packages/pip/__init__.py", line 26, in <module>
    from pip.utils import get_installed_distributions, get_prog
  File "/usr/local/python3.5.2/lib/python3.5/site-packages/pip/utils/__init__.py", line 27, in <module>
    from pip._vendor import pkg_resources
  File "/usr/local/python3.5.2/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3018, in <module>
    @_call_aside
  File "/usr/local/python3.5.2/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3004, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/python3.5.2/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3046, in _initialize_master_t
    dist.activate(replace=False)
  File "/usr/local/python3.5.2/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2578, in activate
    declare_namespace(pkg)
  File "/usr/local/python3.5.2/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2152, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/usr/local/python3.5.2/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2092, in _handle_ns
    _rebuild_mod_path(path, packageName, module)
  File "/usr/local/python3.5.2/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2121, in _rebuild_mod_path
    orig_path.sort(key=position_in_sys_path)
AttributeError: '_NamespacePath' object has no attribute 'sort'

[skorokithakis]

Same here, my pip appears broken with the same error. I'm using a user install of pip:

Traceback (most recent call last):
  File "/home/stavros/.local/bin/pip", line 7, in <module>
    from pip import main
  File "/home/stavros/.local/lib/python3.5/site-packages/pip/__init__.py", line 26, in <module>
    from pip.utils import get_installed_distributions, get_prog
  File "/home/stavros/.local/lib/python3.5/site-packages/pip/utils/__init__.py", line 27, in <module>
    from pip._vendor import pkg_resources
  File "/home/stavros/.local/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3018, in <module>
    @_call_aside
  File "/home/stavros/.local/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3004, in _call_aside
    f(*args, **kwargs)
  File "/home/stavros/.local/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3046, in _initialize_master_working_set
    dist.activate(replace=False)
  File "/home/stavros/.local/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2578, in activate
    declare_namespace(pkg)
  File "/home/stavros/.local/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2152, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/home/stavros/.local/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2092, in _handle_ns
    _rebuild_mod_path(path, packageName, module)
  File "/home/stavros/.local/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2121, in _rebuild_mod_path
    orig_path.sort(key=position_in_sys_path)
AttributeError: '_NamespacePath' object has no attribute 'sort'

[jaraco]

I've noticed that this issue will become more prevelant as more packages (wheels especially) are released using later setuptools versions. I've done more analysis in the upstream issue, but I don't have a fix except to update pkg_resources in pip.

[naisanza]

Getting the same error on the following specs:

• Ubuntu 16.10 Desktop
• apt install python3-pip

Traceback (most recent call last):
  File "/python/v3/Libraries/pip/Upgrade.py", line 4, in <module>
    import pip
  File "/home/eric/.local/lib/python3.5/site-packages/pip/__init__.py", line 26, in <module>
    from pip.utils import get_installed_distributions, get_prog
  File "/home/eric/.local/lib/python3.5/site-packages/pip/utils/__init__.py", line 27, in <module>
    from pip._vendor import pkg_resources
  File "/home/eric/.local/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3018, in <module>
    @_call_aside
  File "/home/eric/.local/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3004, in _call_aside
    f(*args, **kwargs)
  File "/home/eric/.local/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3046, in _initialize_master_working_set
    dist.activate(replace=False)
  File "/home/eric/.local/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2578, in activate
    declare_namespace(pkg)
  File "/home/eric/.local/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2152, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/home/eric/.local/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2092, in _handle_ns
    _rebuild_mod_path(path, packageName, module)
  File "/home/eric/.local/lib/python3.5/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2121, in _rebuild_mod_path
    orig_path.sort(key=position_in_sys_path)
AttributeError: '_NamespacePath' object has no attribute 'sort'

Python code used to upgrade packages:

#!/usr/bin/python3
# -*- coding: utf8 -*-

import pip
import subprocess

try:
    for distribution in pip.get_installed_distributions():
        subprocess.call('python3 -m pip install --upgrade ' + distribution.project_name, shell=True)
except OSError:
    print('Permission denied')

[kelein]

My pip works again after installing conda, see https://conda.io/docs/install/quick.html and https://doc.scrapy.org/en/latest/intro/install.html#anaconda to have a try.