Skip to content

Can't use markers with local wheels after upgrading from 2022.10.12 to 2024.1.0 #6280

@caseyduquettesc

Description

@caseyduquettesc

Issue description

In 2022.10.12, I was vendoring in some locally built wheels that took a while to compile, namely grpcio and pycryptodome. (Pipfile included at end)

vendored_pycryptodome_macosx_12_0_x86_64 = {path = "./vendor/pycryptodome-3.9.9-cp39-cp39-macosx_12_0_x86_64.whl", sys_platform = "== 'darwin'", platform_machine = "== 'x86_64'"}
vendored_pycryptodome_macosx_12_0_arm64 = {path = "./vendor/pycryptodome-3.9.9-cp39-cp39-macosx_12_0_arm64.whl", sys_platform = "== 'darwin'", platform_machine = "== 'arm64'"}

This would resolve the dependencies of the package, but during installation, use the correct wheel file based on the current platform. The lock file would include entries for each package, vendored and non-vendored

"pycryptodome": {
    "hashes": [...],
    "markers": "(sys_platform != 'darwin' or platform_machine != 'arm64') and (sys_platform != 'darwin' or platform_machine != 'x86_64')",
    "version": "==3.9.9"
},
"vendored-grpcio-macosx-12-0-arm64": {
    "path": "./vendor/grpcio-1.50.0-cp39-cp39-macosx_12_0_arm64.whl",
    "platform_machine": "== 'arm64'",
    "sys_platform": "== 'darwin'"
},
"vendored-pycryptodome-macosx-12-0-arm64": {
    "path": "./vendor/pycryptodome-3.9.9-cp39-cp39-macosx_12_0_arm64.whl",
    "platform_machine": "== 'arm64'",
    "sys_platform": "== 'darwin'"
},
"vendored-pycryptodome-macosx-12-0-x86-64": {
    "path": "./vendor/pycryptodome-3.9.9-cp39-cp39-macosx_12_0_x86_64.whl",
    "platform_machine": "== 'x86_64'",
    "sys_platform": "== 'darwin'"
}

There would be some warnings about packages being skipped because they weren't compatible but everything worked correctly.

After upgrading to 2024.1.0, I can't seem to find a way to make local wheels work with markers.

Expected result

I'd expect to be able to apply markers to local wheel files as I could before

Actual result

Locking the same Pipfile fails with the following error

pycryptodome-3.9.9-cp39-cp39-macosx_12_0_x86_64.whl is not a supported wheel on this platform.
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
Could not find a matching version of pycryptodome==3.9.9; (sys_platform != "darwin" or platform_machine != "arm64") and (sys_platform != "darwin" or
platform_machine != "x86_64") for your environment, its dependencies will be skipped.
Could not find a matching version of grpcio==1.50.0; sys_platform != "darwin" or platform_machine != "arm64" for your environment, its dependencies will be
skipped.
Traceback (most recent call last):
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/utils/resolver.py", line 443, in resolve
    results = resolver.resolve(constraints, check_supported_wheels=False)
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/patched/pip/_internal/resolution/resolvelib/resolver.py", line 76,
in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/patched/pip/_internal/resolution/resolvelib/factory.py", line 538,
in collect_root_requirements
    reqs = list(
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/patched/pip/_internal/resolution/resolvelib/factory.py", line 489,
in _make_requirements_from_install_req
    self._fail_if_link_is_unsupported_wheel(ireq.link)
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/patched/pip/_internal/resolution/resolvelib/factory.py", line 145,
in _fail_if_link_is_unsupported_wheel
    raise UnsupportedWheel(msg)
pipenv.patched.pip._internal.exceptions.UnsupportedWheel: pycryptodome-3.9.9-cp39-cp39-macosx_12_0_x86_64.whl is not a supported wheel on this platform.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/resolver.py", line 688, in <module>
    main()
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/resolver.py", line 674, in main
    _main(
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/resolver.py", line 658, in _main
    resolve_packages(
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/resolver.py", line 625, in resolve_packages
    results, resolver = resolve(
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/resolver.py", line 605, in resolve
    return resolve_deps(
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/utils/resolver.py", line 920, in resolve_deps
    results, hashes, internal_resolver = actually_resolve_deps(
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/utils/resolver.py", line 687, in actually_resolve_deps
    resolver.resolve()
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/utils/resolver.py", line 445, in resolve
    raise ResolutionFailure(message=str(e))
pipenv.exceptions.ResolutionFailure: ERROR: pycryptodome-3.9.9-cp39-cp39-macosx_12_0_x86_64.whl is not a supported wheel on this platform.
✘ Locking Failed!
⠸ Locking packages...
Traceback (most recent call last):
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/__main__.py", line 4, in <module>
    cli()
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/cli/options.py", line 52, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/vendor/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/vendor/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/cli/command.py", line 342, in lock
    do_lock(
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/routines/lock.py", line 67, in do_lock
    venv_resolve_deps(
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/utils/resolver.py", line 859, in venv_resolve_deps
    c = resolve(cmd, st, project=project)
  File "/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv/utils/resolver.py", line 723, in resolve
    raise RuntimeError("Failed to lock Pipfile.lock!")
RuntimeError: Failed to lock Pipfile.lock!

If I comment out vendored_pycryptodome_macosx_12_0_x86_64, locking succeeds, but the remaining wheels are missing from the lock file.

Steps to replicate

On MacOS arm64,

pipenv lock --verbose

$ pipenv --support

Pipenv version: '2024.1.0'

Pipenv location: '/Users/casey.duquette/.pyenv/versions/3.9.13/lib/python3.9/site-packages/pipenv'

Python location: '/Users/casey.duquette/.pyenv/versions/3.9.13/bin/python3'

OS Name: 'posix'

User pip version: '24.1.2'

user Python installations found:

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.9.13',
 'os_name': 'posix',
 'platform_machine': 'arm64',
 'platform_python_implementation': 'CPython',
 'platform_release': '23.6.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 23.6.0: Wed Jul 31 20:48:52 PDT '
                     '2024; root:xnu-10063.141.1.700.5~1/RELEASE_ARM64_T6020',
 'python_full_version': '3.9.13',
 'python_version': '3.9',
 'sys_platform': 'darwin'}

System environment variables:

  • SHELL
  • PYENV_HOOK_PATH
  • ITERM_PROFILE
  • COLORTERM
  • PYENV_SHELL
  • XPC_FLAGS
  • TERM_PROGRAM_VERSION
  • rvm_prefix
  • TMUX
  • BYOBU_HIGHLIGHT
  • rvm_alias_expanded
  • TERM_FEATURES
  • __CFBundleIdentifier
  • JAVA_HOME
  • rvm_ruby_file
  • rvm_quiet_flag
  • SSH_AUTH_SOCK
  • rvm_docs_type
  • rvm_gemstone_package_file
  • BYOBU_DATE
  • GOPRIVATE
  • ANDROID_SDK
  • TERM_SESSION_ID
  • MY_RUBY_HOME
  • HOMEBREW_PREFIX
  • rvm_ruby_bits
  • SSH_AGENT_PID
  • RBENV_SHELL
  • BYOBU_CONFIG_DIR
  • EDITOR
  • PYENV_VERSION
  • ANDROID_NDK
  • RUBY_VERSION
  • PWD
  • PYENV_VIRTUALENV_INIT
  • LOGNAME
  • rvm_nightly_flag
  • rvm_version
  • rvm_pretty_print_flag
  • BYOBU_EDITOR
  • rvm_bin_flag
  • COMMAND_MODE
  • ITERM_SESSION_ID
  • rvm_hook
  • HOME
  • LANG
  • BYOBU_ULIMIT
  • BYOBU_TERM
  • VIRTUAL_ENV
  • BYOBU_BACKEND
  • BYOBU_ACCENT
  • rvm_niceness
  • TMPDIR
  • LC_TERMINAL
  • PERL5LIB
  • rvm_silent_flag
  • GOROOT
  • BYOBU_DARK
  • rvm_proxy
  • ANDROID_NDK_HOME
  • PYENV_DIR
  • BYOBU_PAGER
  • NVM_DIR
  • rvm_ruby_make_install
  • rvm_bin_path
  • GEM_PATH
  • GEM_HOME
  • rvm_only_path_flag
  • ANDROID_HOME
  • BYOBU_SED
  • TERM
  • SCALA_HOME
  • PERL_MB_OPT
  • USER
  • TMUX_PANE
  • COLORFGBG
  • HOMEBREW_CELLAR
  • PYENV_VIRTUALENVWRAPPER_PREFER_PYVENV
  • LC_TERMINAL_VERSION
  • PERL_MM_OPT
  • BYOBU_PREFIX
  • SHLVL
  • BYOBU_LIGHT
  • NVM_CD_FLAGS
  • ANDROID_SDK_ROOT
  • BYOBU_TIME
  • HOMEBREW_REPOSITORY
  • SNAP_FROM_SOURCE_ENABLED
  • rvm_ruby_mode
  • BYOBU_CHARMAP
  • BYOBU_WINDOW_NAME
  • VIRTUAL_ENV_PROMPT
  • XPC_SERVICE_NAME
  • BYOBU_READLINK
  • TERMINFO_DIRS
  • HOMEBREW_GITHUB_API_TOKEN
  • BYOBU_DISTRO
  • PYENV_ROOT
  • rvm_ruby_make
  • GONOSUMDB
  • rvm_use_flag
  • PATH
  • rvm_sdk
  • rvm_script_name
  • rvm_gemstone_url
  • IRBRC
  • HOOKS_DISABLED
  • rvm_path
  • OLDPWD
  • GOPATH
  • BYOBU_TTY
  • __CF_USER_TEXT_ENCODING
  • BYOBU_RUN_DIR
  • TERM_PROGRAM
  • PIP_DISABLE_PIP_VERSION_CHECK
  • PYTHONDONTWRITEBYTECODE
  • PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

  • PATH: /Users/casey.duquette/.pyenv/versions/3.6.15_x86_64/bin:/Users/casey.duquette/.pyenv/versions/3.9.13/bin:/Users/casey.duquette/.pyenv/versions/3.8.13/bin:/Users/casey.duquette/.pyenv/versions/3.11.8/bin:/Users/casey.duquette/.pyenv/versions/3.7.14/bin:/Users/casey.duquette/.pyenv/versions/3.8.16_x86_64/bin:/Users/casey.duquette/.pyenv/versions/2.7.18_x86_64/bin:/Users/casey.duquette/.pyenv/versions/3.9.10/bin:/Users/casey.duquette/.pyenv/versions/3.8.16/bin:/Users/casey.duquette/.pyenv/versions/3.7.14_x86_64/bin:/Users/casey.duquette/.pyenv/versions/2.7.18/bin:/Users/casey.duquette/.pyenv/versions/3.9.13/bin:/Users/casey.duquette/.pyenv/libexec:/Users/casey.duquette/.pyenv/plugins/python-build/bin:/Users/casey.duquette/Snapchat/Dev/SC/mobile/phantom/snapci/.venv/bin:/Users/casey.duquette/.svm/current/rt/bin:/Users/casey.duquette/google-cloud-sdk/bin:/Users/casey.duquette/.rbenv/shims:/Users/casey.duquette/.rvm/gems/ruby-3.2.1/bin:/Users/casey.duquette/.rvm/gems/ruby-3.2.1@global/bin:/Users/casey.duquette/.rvm/rubies/ruby-3.2.1/bin:/opt/homebrew/Cellar/pyenv-virtualenv/1.2.4/shims:/Users/casey.duquette/.pyenv/shims:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/usr/local/munki:/usr/local/munkireport:/opt/snapchat/bin:/Users/casey.duquette/.rvm/bin:/Users/casey.duquette/bin:/opt/homebrew/opt/go/libexec/bin:/usr/local/sbin:/Users/casey.duquette/google-cloud-sdk/bin:/Users/casey.duquette/Snapchat/Dev/scripts:/nix/var/nix/profiles/default/bin:/Users/casey.duquette/.pyenv/libexec:/Users/casey.duquette/.pyenv/bin:/Users/casey.duquette/go/bin:/Users/casey.duquette/Library/Android/sdk/platform-tools:/Users/casey.duquette/Library/Android/sdk/ndk:/Users/casey.duquette/.rvm/bin
  • SHELL: /bin/zsh
  • EDITOR: vi
  • LANG: en_US.UTF-8

Contents of Pipfile ('Pipfile'):

[pipenv]
install_search_all_sources = true

[[source]]
name = "registry"
url = "****"
verify_ssl = true

[dev-packages]
mypy = "==0.982"
black = "==22.8.0"
pytest = "==7.1.2"
anybadge = "==1.14.0"
pylint = "==2.15.3"
autoflake = "==1.6.1"
pylint-per-file-ignores = "==1.3.2"

[requires]
# The next line is for IDE use cases and lockfile stability across fix releases
python_version = "3.9"
# IMPORTANT - DO NOT REMOVE - this supports swapping versions between nix/non-nix environments in CI
# python_full_version_nix = "3.9.14"
# python_full_version_ci = "3.9.13"

[packages]
pycryptodome = {version = "==3.9.9", markers = "(sys_platform != 'darwin' or platform_machine != 'arm64') and (sys_platform != 'darwin' or platform_machine != 'x86_64')"}
vendored_pycryptodome_macosx_12_0_x86_64 = {path = "./vendor/pycryptodome-3.9.9-cp39-cp39-macosx_12_0_x86_64.whl", sys_platform = "== 'darwin'", platform_machine = "== 'x86_64'"}
vendored_pycryptodome_macosx_12_0_arm64 = {path = "./vendor/pycryptodome-3.9.9-cp39-cp39-macosx_12_0_arm64.whl", sys_platform = "== 'darwin'", platform_machine = "== 'arm64'"}
grpcio = {version = "==1.50.0", markers = "(sys_platform != 'darwin' or platform_machine != 'arm64')"}
vendored_grpcio_macosx_12_0_arm64 = {path = "./vendor/grpcio-1.50.0-cp39-cp39-macosx_12_0_arm64.whl", sys_platform = "== 'darwin'", platform_machine = "== 'arm64'"}
google-cloud-bigquery = "==3.3.2"
numpy = "<2.0.0"
py7zr = {version = "==0.20.5"}

Metadata

Metadata

Assignees

No one assigned

    Labels

    Status: Awaiting Update ⏳This issue requires more information before assistance can be provided.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions