pip install gcloud fails

[crestenstclair]

on a box in google compute engine:

cresten@blah:/tmp$ pip install gcloud
Downloading/unpacking gcloud
  Downloading gcloud-0.16.0-py2.py3-none-any.whl (568kB): 568kB downloaded
Downloading/unpacking google-gax>=0.12.0 (from gcloud)
  Downloading google-gax-0.12.1.tar.gz
  Running setup.py (path:/tmp/pip_build_cresten/google-gax/setup.py) egg_info for package google-gax

Requirement already satisfied (use --upgrade to upgrade): six in /usr/lib/python2.7/dist-packages (from gcloud)
Downloading/unpacking gax-google-pubsub-v1 (from gcloud)
  Downloading gax-google-pubsub-v1-0.7.10.tar.gz
  Running setup.py (path:/tmp/pip_build_cresten/gax-google-pubsub-v1/setup.py) egg_info for package gax-google-pubsub-v1

    no previously-included directories found matching '.tox'
    warning: no previously-included files matching '*_pb2.py' found anywhere in distribution
Downloading/unpacking googleapis-common-protos (from gcloud)
  Downloading googleapis-common-protos-1.1.0.tar.gz
  Running setup.py (path:/tmp/pip_build_cresten/googleapis-common-protos/setup.py) egg_info for package googleapis-common-protos

Downloading/unpacking httplib2>=0.9.1 (from gcloud)
  Downloading httplib2-0.9.2.zip (210kB): 210kB downloaded
  Running setup.py (path:/tmp/pip_build_cresten/httplib2/setup.py) egg_info for package httplib2

Downloading/unpacking grpcio>=0.14.0 (from gcloud)
  Downloading grpcio-0.15.0.tar.gz (5.3MB): 5.3MB downloaded
  Running setup.py (path:/tmp/pip_build_cresten/grpcio/setup.py) egg_info for package grpcio

    Installed /tmp/pip_build_cresten/grpcio/protobuf-3.0.0b3-py2.7.egg
    Searching for enum34>=1.0.4
    Reading https://pypi.python.org/simple/enum34/
    Best match: enum34 1.1.6
    Downloading https://pypi.python.org/packages/e8/26/a6101edcf724453845c850281b96b89a10dac6bd98edebc82634fccce6a5/enum34-1.1.6.zip#md5=61ad7871532d4ce2d77fac2579237a9e
    Processing enum34-1.1.6.zip
    Writing /tmp/easy_install-1Ep9Wm/enum34-1.1.6/setup.cfg
    Running enum34-1.1.6/setup.py -q bdist_egg --dist-dir /tmp/easy_install-1Ep9Wm/enum34-1.1.6/egg-dist-tmp-Zc_vBk
    zip_safe flag not set; analyzing archive contents...

    Installed /tmp/pip_build_cresten/grpcio/enum34-1.1.6-py2.7.egg
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip_build_cresten/grpcio/setup.py", line 258, in <module>
        test_runner=TEST_RUNNER,
      File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 239, in __init__
        self.fetch_build_eggs(attrs.pop('setup_requires'))
      File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 264, in fetch_build_eggs
        replace_conflicting=True
      File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 620, in resolve
        dist = best[req.key] = env.best_match(req, ws, installer)
      File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 852, in best_match
        dist = working_set.find(req)
      File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 503, in find
        raise VersionConflict(dist,req)     # XXX add more info
    pkg_resources.VersionConflict: (six 1.5.2 (/usr/lib/python2.7/dist-packages), Requirement.parse('six>=1.10'))
    Complete output from command python setup.py egg_info:


Installed /tmp/pip_build_cresten/grpcio/protobuf-3.0.0b3-py2.7.egg

Searching for enum34>=1.0.4

Reading https://pypi.python.org/simple/enum34/

Best match: enum34 1.1.6

Downloading https://pypi.python.org/packages/e8/26/a6101edcf724453845c850281b96b89a10dac6bd98edebc82634fccce6a5/enum34-1.1.6.zip#md5=61ad7871532d4ce2d77fac2579237a9e

Processing enum34-1.1.6.zip

Writing /tmp/easy_install-1Ep9Wm/enum34-1.1.6/setup.cfg

Running enum34-1.1.6/setup.py -q bdist_egg --dist-dir /tmp/easy_install-1Ep9Wm/enum34-1.1.6/egg-dist-tmp-Zc_vBk

zip_safe flag not set; analyzing archive contents...



Installed /tmp/pip_build_cresten/grpcio/enum34-1.1.6-py2.7.egg

Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/tmp/pip_build_cresten/grpcio/setup.py", line 258, in <module>

    test_runner=TEST_RUNNER,

  File "/usr/lib/python2.7/distutils/core.py", line 111, in setup

    _setup_distribution = dist = klass(attrs)

  File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 239, in __init__

    self.fetch_build_eggs(attrs.pop('setup_requires'))

  File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 264, in fetch_build_eggs

    replace_conflicting=True

  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 620, in resolve

    dist = best[req.key] = env.best_match(req, ws, installer)

  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 852, in best_match

    dist = working_set.find(req)

  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 503, in find

    raise VersionConflict(dist,req)     # XXX add more info

pkg_resources.VersionConflict: (six 1.5.2 (/usr/lib/python2.7/dist-packages), Requirement.parse('six>=1.10'))

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_cresten/grpcio
Storing debug log for failure in /home/cresten/.pip/pip.log
cresten@blah:/tmp$ pip install --upgrade gcloud
Downloading/unpacking gcloud
  Downloading gcloud-0.16.0-py2.py3-none-any.whl (568kB): 568kB downloaded
Downloading/unpacking google-gax>=0.12.0 (from gcloud)
  Downloading google-gax-0.12.1.tar.gz
  Running setup.py (path:/tmp/pip_build_cresten/google-gax/setup.py) egg_info for package google-gax

Downloading/unpacking six from https://pypi.python.org/packages/c8/0a/b6723e1bc4c516cb687841499455a8505b44607ab535be01091c0f24f079/six-1.10.0-py2.py3-none-any.whl#md5=3ab558cf5d4f7a72611d59a81a315dc8 (from gcloud)
  Downloading six-1.10.0-py2.py3-none-any.whl
Downloading/unpacking gax-google-pubsub-v1 (from gcloud)
  Downloading gax-google-pubsub-v1-0.7.10.tar.gz
  Running setup.py (path:/tmp/pip_build_cresten/gax-google-pubsub-v1/setup.py) egg_info for package gax-google-pubsub-v1

    no previously-included directories found matching '.tox'
    warning: no previously-included files matching '*_pb2.py' found anywhere in distribution
Downloading/unpacking googleapis-common-protos (from gcloud)
  Downloading googleapis-common-protos-1.1.0.tar.gz
  Running setup.py (path:/tmp/pip_build_cresten/googleapis-common-protos/setup.py) egg_info for package googleapis-common-protos

Downloading/unpacking httplib2>=0.9.1 (from gcloud)
  Downloading httplib2-0.9.2.zip (210kB): 210kB downloaded
  Running setup.py (path:/tmp/pip_build_cresten/httplib2/setup.py) egg_info for package httplib2

Downloading/unpacking grpcio>=0.14.0 (from gcloud)
  Downloading grpcio-0.15.0.tar.gz (5.3MB): 5.3MB downloaded
  Running setup.py (path:/tmp/pip_build_cresten/grpcio/setup.py) egg_info for package grpcio

    Installed /tmp/pip_build_cresten/grpcio/protobuf-3.0.0b3-py2.7.egg
    Searching for enum34>=1.0.4
    Reading https://pypi.python.org/simple/enum34/
    Best match: enum34 1.1.6
    Downloading https://pypi.python.org/packages/e8/26/a6101edcf724453845c850281b96b89a10dac6bd98edebc82634fccce6a5/enum34-1.1.6.zip#md5=61ad7871532d4ce2d77fac2579237a9e
    Processing enum34-1.1.6.zip
    Writing /tmp/easy_install-PQk9ld/enum34-1.1.6/setup.cfg
    Running enum34-1.1.6/setup.py -q bdist_egg --dist-dir /tmp/easy_install-PQk9ld/enum34-1.1.6/egg-dist-tmp-YUpuze
    zip_safe flag not set; analyzing archive contents...

    Installed /tmp/pip_build_cresten/grpcio/enum34-1.1.6-py2.7.egg
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip_build_cresten/grpcio/setup.py", line 258, in <module>
        test_runner=TEST_RUNNER,
      File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 239, in __init__
        self.fetch_build_eggs(attrs.pop('setup_requires'))
      File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 264, in fetch_build_eggs
        replace_conflicting=True
      File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 620, in resolve
        dist = best[req.key] = env.best_match(req, ws, installer)
      File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 852, in best_match
        dist = working_set.find(req)
      File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 503, in find
        raise VersionConflict(dist,req)     # XXX add more info
    pkg_resources.VersionConflict: (six 1.5.2 (/usr/lib/python2.7/dist-packages), Requirement.parse('six>=1.10'))
    Complete output from command python setup.py egg_info:


Installed /tmp/pip_build_cresten/grpcio/protobuf-3.0.0b3-py2.7.egg

Searching for enum34>=1.0.4

Reading https://pypi.python.org/simple/enum34/

Best match: enum34 1.1.6

Downloading https://pypi.python.org/packages/e8/26/a6101edcf724453845c850281b96b89a10dac6bd98edebc82634fccce6a5/enum34-1.1.6.zip#md5=61ad7871532d4ce2d77fac2579237a9e

Processing enum34-1.1.6.zip

Writing /tmp/easy_install-PQk9ld/enum34-1.1.6/setup.cfg

Running enum34-1.1.6/setup.py -q bdist_egg --dist-dir /tmp/easy_install-PQk9ld/enum34-1.1.6/egg-dist-tmp-YUpuze

zip_safe flag not set; analyzing archive contents...



Installed /tmp/pip_build_cresten/grpcio/enum34-1.1.6-py2.7.egg

Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/tmp/pip_build_cresten/grpcio/setup.py", line 258, in <module>

    test_runner=TEST_RUNNER,

  File "/usr/lib/python2.7/distutils/core.py", line 111, in setup

    _setup_distribution = dist = klass(attrs)

  File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 239, in __init__

    self.fetch_build_eggs(attrs.pop('setup_requires'))

  File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 264, in fetch_build_eggs

    replace_conflicting=True

  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 620, in resolve

    dist = best[req.key] = env.best_match(req, ws, installer)

  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 852, in best_match

    dist = working_set.find(req)

  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 503, in find

    raise VersionConflict(dist,req)     # XXX add more info

pkg_resources.VersionConflict: (six 1.5.2 (/usr/lib/python2.7/dist-packages), Requirement.parse('six>=1.10'))

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_cresten/grpcio
Storing debug log for failure in /home/cresten/.pip/pip.log

[theacodes]

Can you tell us which OS you're using?

pip is refusing to install a newer version of six because the OS has installed a particular version in dist-packages.

[crestenstclair]

ubuntu 14.04

[theacodes]

Summoning @nathanielmanistaatgoogle: grpc fails on ubuntu 14.04 D:

[theacodes]

@csaintc a workaround for now is to use a virtualenv.

[nathanielmanistaatgoogle]

I suspect @soltanmm and @kpayson64 will want to see this.

What's the root cause? Is it a problem that grpcio requires six>=1.10? If so, why?

[theacodes]

pip won't install anything in site-packages if it conflicts with a package in dist-packages. dist-packages is for the OS package manager (apt/yum, etc.) and site-packages is for pip. Since grpcio requires a newer version of six, pip won't install it and "break" the system.

[kpayson64]

It seems like six>=1.10 was the version we required since we started requiring six:
grpc/grpc@154b0ee

We might try six>=1.5.2 for the 0.15.1 release.

Are we only seeing this because now because gcloud just added a hard dependency to grpcio?
#1879

[theacodes]

Yeah, probably.

[theacodes]

Actually, @csaintc I just noticed you ran this as non-root. Can you try re-running as root? It's expected that installing system-wide Python packages as non-root will fail, although pip's error isn't very helpful.

[theacodes]

Nevermind, I can confirm this fails as root as well.

[theacodes]

FWIW:

root@8d3359f13993:/# pip install --upgrade six
Downloading/unpacking six from https://pypi.python.org/packages/c8/0a/b6723e1bc4c516cb687841499455a8505b44607ab535be01091c0f24f079/six-1.10.0-py2.py3-none-any.whl#md5=3ab558cf5d4f7a72611d59a81a315dc8
  Downloading six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six
  Found existing installation: six 1.5.2
    Not uninstalling six at /usr/lib/python2.7/dist-packages, owned by OS
Successfully installed six
Cleaning up...

Note Not uninstalling six at /usr/lib/python2.7/dist-packages, owned by OS. Debian has a weird patch to pip that causes fun stuff to happen.

You can get gcloud to work by doing this (as root):

pip install --upgrade six
pip install gcloud

[soltanmm]

@kpayson64 re six>=1.5.2: grpc/grpc#5284 (comment)

EDIT: This isn't me saying, "no no no no no no no," it's me saying, "urk - if you can make pip act like a CSP all the more power to ya."