Version conflict ulrlib while running sagify init

[nsorros]

To reproduce in my laptop which uses Python 3.7.9

python -m venv venv
source venv/bin/activate
pip install sagemaker
pip install sagify
sagify init

This throws

Traceback (most recent call last):
  File "/Users/nicksorros/code/test/sagemaker/sagify/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 584, in _build_master
    ws.require(__requires__)
  File "/Users/nicksorros/code/test/sagemaker/sagify/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 901, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/Users/nicksorros/code/test/sagemaker/sagify/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 792, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (urllib3 1.24.3 (/Users/nicksorros/code/test/sagemaker/sagify/venv/lib/python3.7/site-packages), Requirement.parse('urllib3<1.27,>=1.25.4; python_version != "3.4"'), {'botocore'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/nicksorros/code/test/sagemaker/sagify/venv/bin/sagify", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/Users/nicksorros/code/test/sagemaker/sagify/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3261, in <module>
    @_call_aside
  File "/Users/nicksorros/code/test/sagemaker/sagify/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3245, in _call_aside
    f(*args, **kwargs)
  File "/Users/nicksorros/code/test/sagemaker/sagify/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3274, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/Users/nicksorros/code/test/sagemaker/sagify/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 586, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/Users/nicksorros/code/test/sagemaker/sagify/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 599, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/Users/nicksorros/code/test/sagemaker/sagify/venv/lib/python3.7/site-packages/pkg_resources/__init__.py", line 792, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (urllib3 1.24.3 (/Users/nicksorros/code/test/sagemaker/sagify/venv/lib/python3.7/site-packages), Requirement.parse('urllib3<1.27,>=1.25.4; python_version != "3.4"'), {'botocore'})

Happy to provide more details about my environment. My current theory is that it either has to do with my Python version as I have tried in AWS with python 3.6 and it works or that it has to do with sagemaker version.

[pm3310]

Hey @nsorros thanks for the issue. You don't need to execute pip install sagemaker. When you run pip install sagify, it installs a specific sagemaker version for you. Can you give it a try again and let me know?

[nsorros]

Interestingly the problem persists. This the output from the installation in case it helps

Collecting sagify
  Using cached sagify-0.20.9.tar.gz (40 kB)
Collecting boto3
  Using cached boto3-1.16.63-py2.py3-none-any.whl (130 kB)
Collecting click<7.0.99,>=7.0
  Using cached Click-7.0-py2.py3-none-any.whl (81 kB)
Collecting docker<3.7.99,>=3.7.0
  Using cached docker-3.7.3-py2.py3-none-any.whl (134 kB)
Collecting flask<1.1.99,>=1.1.0
  Using cached Flask-1.1.2-py2.py3-none-any.whl (94 kB)
Collecting paramiko<2.4.99,>=2.4.2
  Using cached paramiko-2.4.3-py2.py3-none-any.whl (193 kB)
Collecting pathlib2<2.3.99,>=2.3.0
  Using cached pathlib2-2.3.5-py2.py3-none-any.whl (18 kB)
Collecting requests<2.20.99,>=2.20.0
  Using cached requests-2.20.1-py2.py3-none-any.whl (57 kB)
Collecting sagemaker<1.50.99,>=1.50.0
  Using cached sagemaker-1.50.18.post0.tar.gz (294 kB)
Collecting six<1.11.99,>=1.10
  Using cached six-1.11.0-py2.py3-none-any.whl (10 kB)
Collecting future<0.17.99,>=0.16.0
  Using cached future-0.17.1.tar.gz (829 kB)
Collecting jmespath<1.0.0,>=0.7.1
  Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting s3transfer<0.4.0,>=0.3.0
  Using cached s3transfer-0.3.4-py2.py3-none-any.whl (69 kB)
Collecting botocore<1.20.0,>=1.19.63
  Using cached botocore-1.19.63-py2.py3-none-any.whl (7.2 MB)
Collecting websocket-client>=0.32.0
  Using cached websocket_client-0.57.0-py2.py3-none-any.whl (200 kB)
Collecting docker-pycreds>=0.4.0
  Using cached docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting itsdangerous>=0.24
  Using cached itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting Jinja2>=2.10.1
  Downloading Jinja2-2.11.3-py2.py3-none-any.whl (125 kB)
     |████████████████████████████████| 125 kB 584 kB/s
Collecting Werkzeug>=0.15
  Using cached Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
Collecting pynacl>=1.0.1
  Using cached PyNaCl-1.4.0-cp35-abi3-macosx_10_10_x86_64.whl (380 kB)
Collecting bcrypt>=3.1.3
  Using cached bcrypt-3.2.0-cp36-abi3-macosx_10_9_x86_64.whl (31 kB)
Collecting pyasn1>=0.1.7
  Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting cryptography>=1.5
  Using cached cryptography-3.3.1-cp36-abi3-macosx_10_10_x86_64.whl (1.8 MB)
Collecting idna<2.8,>=2.5
  Using cached idna-2.7-py2.py3-none-any.whl (58 kB)
Collecting chardet<3.1.0,>=3.0.2
  Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting urllib3<1.25,>=1.21.1
  Using cached urllib3-1.24.3-py2.py3-none-any.whl (118 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2020.12.5-py2.py3-none-any.whl (147 kB)
Collecting numpy>=1.9.0
  Downloading numpy-1.20.0-cp37-cp37m-macosx_10_9_x86_64.whl (16.0 MB)
     |████████████████████████████████| 16.0 MB 2.2 MB/s
Collecting protobuf>=3.1
  Using cached protobuf-3.14.0-cp37-cp37m-macosx_10_9_x86_64.whl (1.0 MB)
Collecting scipy>=0.19.0
  Using cached scipy-1.6.0-cp37-cp37m-macosx_10_9_x86_64.whl (30.7 MB)
Collecting protobuf3-to-dict>=0.1.5
  Using cached protobuf3-to-dict-0.1.5.tar.gz (3.5 kB)
Collecting smdebug-rulesconfig==0.1.2
  Using cached smdebug_rulesconfig-0.1.2-py2.py3-none-any.whl (10 kB)
Collecting importlib-metadata>=1.4.0
  Using cached importlib_metadata-3.4.0-py3-none-any.whl (10 kB)
Collecting packaging>=20.0
  Using cached packaging-20.9-py2.py3-none-any.whl (40 kB)
Collecting python-dateutil<3.0.0,>=2.1
  Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting MarkupSafe>=0.23
  Using cached MarkupSafe-1.1.1-cp37-cp37m-macosx_10_9_x86_64.whl (16 kB)
Collecting cffi>=1.4.1
  Using cached cffi-1.14.4-cp37-cp37m-macosx_10_9_x86_64.whl (176 kB)
Collecting zipp>=0.5
  Using cached zipp-3.4.0-py3-none-any.whl (5.2 kB)
Collecting typing-extensions>=3.6.4; python_version < "3.8"
  Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Collecting pyparsing>=2.0.2
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting pycparser
  Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Using legacy setup.py install for sagify, since package 'wheel' is not installed.
Using legacy setup.py install for sagemaker, since package 'wheel' is not installed.
Using legacy setup.py install for future, since package 'wheel' is not installed.
Using legacy setup.py install for protobuf3-to-dict, since package 'wheel' is not installed.
ERROR: botocore 1.19.63 has requirement urllib3<1.27,>=1.25.4; python_version != "3.4", but you'll have urllib3 1.24.3 which is incompatible.
Installing collected packages: jmespath, urllib3, six, python-dateutil, botocore, s3transfer, boto3, click, idna, chardet, certifi, requests, websocket-client, docker-pycreds, docker, itsdangerous, MarkupSafe, Jinja2, Werkzeug, flask, pycparser, cffi, pynacl, bcrypt, pyasn1, cryptography, paramiko, pathlib2, numpy, protobuf, scipy, protobuf3-to-dict, smdebug-rulesconfig, zipp, typing-extensions, importlib-metadata, pyparsing, packaging, sagemaker, future, sagify
    Running setup.py install for protobuf3-to-dict ... done
    Running setup.py install for sagemaker ... done
    Running setup.py install for future ... done
    Running setup.py install for sagify ... done
Successfully installed Jinja2-2.11.3 MarkupSafe-1.1.1 Werkzeug-1.0.1 bcrypt-3.2.0 boto3-1.16.63 botocore-1.19.63 certifi-2020.12.5 cffi-1.14.4 chardet-3.0.4 click-7.0 cryptography-3.3.1 docker-3.7.3 docker-pycreds-0.4.0 flask-1.1.2 future-0.17.1 idna-2.7 importlib-metadata-3.4.0 itsdangerous-1.1.0 jmespath-0.10.0 numpy-1.20.0 packaging-20.9 paramiko-2.4.3 pathlib2-2.3.5 protobuf-3.14.0 protobuf3-to-dict-0.1.5 pyasn1-0.4.8 pycparser-2.20 pynacl-1.4.0 pyparsing-2.4.7 python-dateutil-2.8.1 requests-2.20.1 s3transfer-0.3.4 sagemaker-1.50.18.post0 sagify-0.20.9 scipy-1.6.0 six-1.11.0 smdebug-rulesconfig-0.1.2 typing-extensions-3.7.4.3 urllib3-1.24.3 websocket-client-0.57.0 zipp-3.4.0
WARNING: You are using pip version 20.1.1; however, version 21.0.1 is available.

[nsorros]

Upgrading pip seems to resolve it. Happy for this to close but keeping it open in case you want to investigate the exact cause.

[pm3310]

@nsorros thank you for the update. I'll close it for now as I couldn't reproduce it with your previous pip version :-(