Editable dependency requirements sometimes override top-level ones

[greysteil]

Issue description

I'm not 100% sure how this is happening, but when locking a Pipfile that includes an editable requirement, sometimes the requirements from that dependency are "preferred" (i.e., overwrite) the top-level requirements for dependencies.

An example is probably the best illustration:

Pipfile

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true

[packages]
requests = { editable = true, git = 'https://github.com/requests/requests.git', ref = 'v2.18.4' }
urllib3 = "==1.21.1"

Result from calling pyenv exec pipenv lock:

{
    "_meta": {
        "hash": {
            "sha256": "71c49547d80da72c513f8e15ba6c8c070007d930c22dff508902f491b7665107"
        },
        "pipfile-spec": 6,
        "requires": {},
        "sources": [
            {
                "url": "https://pypi.python.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "certifi": {
            "hashes": [
                "sha256:339dc09518b07e2fa7eda5450740925974815557727d6bd35d319c1524a04a4c",
                "sha256:6d58c986d22b038c8c0df30d639f23a3e6d172a05c3583e766f4c0b785c0986a"
            ],
            "version": "==2018.10.15"
        },
        "chardet": {
            "hashes": [
                "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
                "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
            ],
            "version": "==3.0.4"
        },
        "idna": {
            "hashes": [
                "sha256:2c6a5de3089009e3da7c5dde64a141dbc8551d5b7f6cf4ed7c2568d0cc520a8f",
                "sha256:8c7309c718f94b3a625cb648ace320157ad16ff131ae0af362c9f21b80ef6ec4"
            ],
            "version": "==2.6"
        },
        "requests": {
            "editable": true,
            "git": "https://github.com/requests/requests.git",
            "ref": "a3d7cf3f27e74c28ef30f01e9f2e483570ab042e"
        },
        "urllib3": {
            "hashes": [
                "sha256:06330f386d6e4b195fbfc736b297f58c5a892e4440e54d294d7004e3a9bbea1b",
                "sha256:cc44da8e1145637334317feebd728bd869a35285b93cbb4cca2577da7e62db4f"
            ],
            "version": "==1.22"
        }
    },
    "develop": {}
}

Expected result

The version of urllib3 should be 1.21.1.

Actual result

The version of urllib3 is 1.22.

Steps to replicate

Included in description. Happens on 3ce1394.

---

$ pipenv --support

Pipenv version: '2018.11.15.dev0'

Pipenv location: '/Users/greysteil/code/dependabot-script/.bundle/bundler/gems/dependabot-core-df7316130ca8/helpers/python/src/pipenv/pipenv'

Python location: '/Users/greysteil/.pyenv/versions/3.6.6/bin/python'

Python installations found:

• 3.7.0: /Users/greysteil/.pyenv/versions/3.7.0/bin/python3
• 3.7.0: /Users/greysteil/.pyenv/versions/3.7.0/bin/python3.7m
• 3.6.7: /Users/greysteil/.pyenv/versions/3.6.7/bin/python3
• 3.6.7: /Users/greysteil/.pyenv/versions/3.6.7/bin/python3.6m
• 3.6.6: /Users/greysteil/.pyenv/versions/3.6.6/bin/python3
• 3.6.6: /Users/greysteil/.pyenv/versions/3.6.6/bin/python3.6m
• 3.6.5: /Users/greysteil/.pyenv/versions/3.6.5/bin/python3
• 3.6.5: /Users/greysteil/.pyenv/versions/3.6.5/bin/python3.6m
• 3.6.5: /usr/local/bin/python3
• 3.6.5: /usr/local/bin/python3.6m
• 3.6.2: /Users/greysteil/.pyenv/versions/3.6.2/bin/python3
• 3.6.2: /Users/greysteil/.pyenv/versions/3.6.2/bin/python3.6m
• 3.6.1: /Users/greysteil/.pyenv/versions/3.6.1/bin/python3
• 3.6.1: /Users/greysteil/.pyenv/versions/3.6.1/bin/python3.6m
• 3.5.0: /Users/greysteil/.pyenv/versions/3.5.0/bin/python3
• 3.5.0: /Users/greysteil/.pyenv/versions/3.5.0/bin/python3.5m
• 2.7.15: /Users/greysteil/.pyenv/versions/2.7.15/bin/python
• 2.7.14: /Users/greysteil/.pyenv/versions/2.7.14/bin/python
• 2.7.10: /usr/bin/python
• 2.7.10: /usr/bin/pythonw
• 2.7.10: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.6',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '17.7.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 17.7.0: Wed Oct 10 23:06:14 PDT '
                     '2018; root:xnu-4570.71.13~1/RELEASE_X86_64',
 'python_full_version': '3.6.6',
 'python_version': '3.6',
 'sys_platform': 'darwin'}

System environment variables:

• TERM_PROGRAM
• PYENV_ROOT
• TERM
• SHELL
• CLICOLOR
• TMPDIR
• Apple_PubSub_Socket_Render
• TERM_PROGRAM_VERSION
• TERM_SESSION_ID
• PYENV_VERSION
• USER
• SSH_AUTH_SOCK
• PYENV_DIR
• LSCOLORS
• PATH
• PWD
• EDITOR
• LANG
• PYENV_HOOK_PATH
• XPC_FLAGS
• RBENV_SHELL
• XPC_SERVICE_NAME
• HOME
• SHLVL
• LOGNAME
• __CF_USER_TEXT_ENCODING
• PIP_DISABLE_PIP_VERSION_CHECK
• PYTHONDONTWRITEBYTECODE
• PIP_SHIMS_BASE_MODULE
• PIP_PYTHON_PATH
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /Users/greysteil/.pyenv/versions/3.6.6/bin:/usr/local/Cellar/pyenv/1.2.8/libexec:/Users/greysteil/.pyenv/plugins/pyenv-virtualenv/bin:/Users/greysteil/.pyenv/plugins/pyenv-update/bin:/Users/greysteil/.pyenv/plugins/pyenv-installer/bin:/Users/greysteil/.pyenv/plugins/pyenv-doctor/bin:/Users/greysteil/.poetry/bin:/Users/greysteil/.config/yarn/global/node_modules/.bin:/Users/greysteil/.poetry/bin::/Users/greysteil/go/bin::/usr/local/opt/erlang@20/bin:/Users/greysteil/.pyenv/bin:/Users/greysteil/.cargo/bin:/usr/local/heroku/bin:/Users/greysteil/.rbenv/shims:/usr/local/bin:./node_modules/.bin:.bundle/binstubs:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/usr/local/go/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/usr/local/git/bin:/usr/local/sbin
• SHELL: /bin/bash
• EDITOR: subl -w
• LANG: en_GB.UTF-8
• PWD: /Users/greysteil/code/python-test

---

Contents of Pipfile ('/Users/greysteil/code/python-test/Pipfile'):

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true

[packages]
requests = { editable = true, git = 'https://github.com/requests/requests.git', ref = 'v2.18.4' }
urllib3 = "==1.21.1"

Contents of Pipfile.lock ('/Users/greysteil/code/python-test/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "71c49547d80da72c513f8e15ba6c8c070007d930c22dff508902f491b7665107"
        },
        "pipfile-spec": 6,
        "requires": {},
        "sources": [
            {
                "url": "https://pypi.python.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "certifi": {
            "hashes": [
                "sha256:339dc09518b07e2fa7eda5450740925974815557727d6bd35d319c1524a04a4c",
                "sha256:6d58c986d22b038c8c0df30d639f23a3e6d172a05c3583e766f4c0b785c0986a"
            ],
            "version": "==2018.10.15"
        },
        "chardet": {
            "hashes": [
                "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
                "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
            ],
            "version": "==3.0.4"
        },
        "idna": {
            "hashes": [
                "sha256:2c6a5de3089009e3da7c5dde64a141dbc8551d5b7f6cf4ed7c2568d0cc520a8f",
                "sha256:8c7309c718f94b3a625cb648ace320157ad16ff131ae0af362c9f21b80ef6ec4"
            ],
            "version": "==2.6"
        },
        "requests": {
            "editable": true,
            "git": "https://github.com/requests/requests.git",
            "ref": "a3d7cf3f27e74c28ef30f01e9f2e483570ab042e"
        },
        "urllib3": {
            "hashes": [
                "sha256:06330f386d6e4b195fbfc736b297f58c5a892e4440e54d294d7004e3a9bbea1b",
                "sha256:cc44da8e1145637334317feebd728bd869a35285b93cbb4cca2577da7e62db4f"
            ],
            "version": "==1.22"
        }
    },
    "develop": {}
}

[techalchemy]

It’s definitely happening, we don’t always trust the resolver on editavke dependencies so we don’t pass no-deps during install. These get installed last. Question is, is this a regression?

[techalchemy]

Basically if it is, I’d love to trust what we resolved.

[greysteil]

It's a regression from 2018.10.13 - I get the correct version of urllib3 when running that. It had already regressed in 2018.11.14, though.

[jpcope]

We've had this happen to us still on release 2018.11.26

I wish I had an example I could share, but I do not because the repos involved are private and were accessed via a gitconfig url rewrite that adds a private github auth token ( if that helps identify where it is still a problem )