Six issue when installing package

[securitybites]

When trying to install awscli I get the following error.

admins-MacBook-Pro-5:~ user$ pip install awscli
Collecting awscli
  Using cached awscli-1.8.9-py2.py3-none-any.whl
Collecting botocore==1.2.7 (from awscli)
  Using cached botocore-1.2.7-py2.py3-none-any.whl
Collecting rsa<=3.3.0,>=3.1.2 (from awscli)
  Using cached rsa-3.2-py2.py3-none-any.whl
Collecting colorama<=0.3.3,>=0.2.5 (from awscli)
Collecting docutils>=0.10 (from awscli)
Requirement already satisfied (use --upgrade to upgrade): jmespath<1.0.0,>=0.7.1 in /Library/Python/2.7/site-packages (from botocore==1.2.7->awscli)
Collecting python-dateutil<3.0.0,>=2.1 (from botocore==1.2.7->awscli)
  Using cached python_dateutil-2.4.2-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3 (from rsa<=3.3.0,>=3.1.2->awscli)
  Using cached pyasn1-0.1.9-py2.py3-none-any.whl
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore==1.2.7->awscli)
  Using cached six-1.9.0-py2.py3-none-any.whl
Installing collected packages: six, python-dateutil, docutils, botocore, pyasn1, rsa, colorama, awscli
  Found existing installation: six 1.4.1
    DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip/commands/install.py", line 311, in run
    root=options.root_path,
  File "/Library/Python/2.7/site-packages/pip/req/req_set.py", line 640, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip/req/req_install.py", line 716, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip/req/req_uninstall.py", line 125, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip/utils/__init__.py", line 315, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
    copystat(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
    os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/var/folders/6t/h404bjcd5tb_4q86tpv_251rv_0h0j/T/pip-sYsqDS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'

Other user have mentioned the issue and point to pip:

aws/aws-cli#1522

https://bitbucket.org/gutworth/six/issues/134/uninstall-error-operation-not-permitted

[dstufft]

You're on El-Capitan right?

[moeriki]

I have this issue when installing AWS-SDK, which depends on six.

On El-Capitan indeed.

[dstufft]

This is because OS X El Capitan ships with six 1.4.1 installed already and when it attempts to uninstall it (because awscli depends on botocore, botocore depends on python-dateutil, and python-dateutil depends on six >= 1.5) it doesn't have permission to do so because System Integrity Protection doesn't allow even root to modify those directories.

Ideally, pip should just skip uninstalling those items since they aren't installed to site-packages they are installed to a special Apple directory. However, even if pip skips uninstalling those items and installs six into site-packages we'll hit another bug where Apple puts their pre-installed stuff earlier in the sys.path than site-packages. I've talked to Apple about this and I'm not sure if they're going to do anything about it or not.

[arubdesu]

This affects many other packages which rely on Six, pardon my obliviousness, but is there any way to tell pip to either 1. install an updated version to /Library's site-packages and be pointed to prefer that one (instead of saying 'requirement satisfied! and exiting) or 2. ignore the version found in /System on 10.11+?

[dstufft]

pip install --ignore-installed six

[arubdesu]

$ sudo -H pip install --ignore-installed six
Collecting six
  Using cached six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six
Successfully installed six-1.4.1

Sure, but pip won't then prefer that upgraded version of Six when performing sudo installs:

sudo -H pip install mock
Requirement already satisfied (use --upgrade to upgrade): mock in /Library/Python/2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): funcsigs in /Library/Python/2.7/site-packages (from mock)
Requirement already satisfied (use --upgrade to upgrade): pbr>=0.11 in /Library/Python/2.7/site-packages (from mock)
Collecting six>=1.7 (from mock)
  Downloading six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six
  Found existing installation: six 1.4.1
    DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/commands/install.py", line 311, in run
    root=options.root_path,
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/req/req_set.py", line 640, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/req/req_install.py", line 716, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/req/req_uninstall.py", line 125, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/utils/__init__.py", line 315, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
    copystat(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
    os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/tmp/pip-sy_6uC-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'

[arubdesu]

...
Sorry about that, for some reason it's working after... running it a second time? Needed to --ignore-installed mock as well.

[arubdesu]

Ah, so this is less a pip issue to address (although it seems like --ignore-installed is going to get very popular) than the user changing their path appropriately. (You'll just get all the support requests. Thanks, Apple!)
As per http://peak.telecommunity.com/DevCenter/setuptools#development-mode you can set a .pth file system-wide to make sure that directory is consulted first. I'll compose a radar to Apple so they can remove their heads from their butts about this.

[arubdesu]

...Shouldn't there be some other, more permanent fix? easy_install has a .pth file, so there is some precedent here. I've documented what I think is the best near-term solution here: http://apple.stackexchange.com/a/210021/216

[reganjohnson]

THANKS! The following allowed it to install correctly.

pip install awsebcli --upgrade --ignore-installed six

Running on OSX El Capitan

$ eb --version
EB CLI 3.5.4 (Python 2.7.1)

[dstufft]

@arubdesu I'm hoping the more permanent fix will be to have Apple just fix this, which I am talking to them about.

[propupul]

Hello all!
I came across the same problem(thank goodness for this post!) while trying to install virtualenvwrapper. As all of you, and lastly @reganjohnson suggested --upgrade --ignore-installed six will work.
I did
sudo pip install virtualenvwrapper --upgrade --ignore-installed six

Running El Capitan as well

P.S: Please forgive me if I'm not posting this correctly I'm a complete noob to github :(

[nalindak]

sudo pip install virtualenvwrapper --upgrade --ignore-installed six

Did work for me! thanks guys