Python version mismatch between venv and pipfile upon creation of venv (no preexisting Pipfile)

[bgilliland]

Issue description

When creating a new virtual environment ( pipenv install ) in an empty directory (ie no Pipfile or Pipfile.lock files) the Python version shown in the Terminal output does not match the python_version in the Pipfile.

The terminal output shows the venv being created with whatever the newest version of python is on my system (3.11.3 for me). The python_version in the Pipfile is not this version, though. Rather it is whatever my pyenv global setting is. I've tinkered with switching the global version to see if the Pipfile would always refer to it in python_version and it indeed does.

My hypothesis is that pyenv and pipenv are not aligned on how to create the virtual environment and how the Pipfile should be created.

Expected result

python_version in Pipfile to match the version that the venv was created to use (which is the newest version of python on my system)

Actual result

python_version is set to the global Python version as set in pyenv while venv is created to use newest version of python on my machine

Steps to replicate

Go to empty directory, confirm it does not have Pipfiles:

> cd path/to/project
~/path/to/project

> ls

Confirm global python version used by pyenv:

> pyenv global
3.9.16

Install the venv:

> pipenv install
Pipfile: /Users/blakegilliland/path/to/project
Using /opt/homebrew/bin/python3 (3.11.3) to create virtualenv...
⠴ Creating virtual environment...created virtual environment CPython3.11.3.final.0-64 in 357ms
  creator CPython3Posix(dest=/Users/blakegilliland/.local/share/virtualenvs/dummyproj-X3m8E-g_, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/blakegilliland/Library/Application Support/virtualenv)
    added seed packages: pip==23.1, setuptools==67.6.1, wheel==0.40.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /Users/blakegilliland/.local/share/virtualenvs/dummyproj-X3m8E-g_
Creating a Pipfile for this project...
Pipfile.lock not found, creating...
Locking [packages] dependencies...
Locking [dev-packages] dependencies...
Updated Pipfile.lock (fedbd2ab7afd84cf16f128af0619749267b62277b4cb6989ef16d4bef6e4eef2)!
Installing dependencies from Pipfile.lock (e4eef2)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

Confirm venv is using 3.11.3 to run python:

> pipenv run python -V
Python 3.11.3

Pipfile:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]

[dev-packages]

[requires]
python_version = "3.9"

---

Pipenv version: '2023.3.20'

Pipenv location: '/Users/blakegilliland/.pyenv/versions/3.9.16/lib/python3.9/site-packages/pipenv'

Python location: '/Users/blakegilliland/.pyenv/versions/3.9.16/bin/python3.9'

OS Name: 'posix'

User pip version: '23.0.1'

user Python installations found:

• 3.11.3: /opt/homebrew/bin/python3
• 3.10.11: /Users/blakegilliland/.pyenv/versions/3.10.11/bin/python3
• 3.9.16: /Users/blakegilliland/.pyenv/versions/3.9.16/bin/python3
• 3.9.16: /Users/blakegilliland/.pyenv/versions/3.9.16/bin/python3
• 3.9.16: /Users/blakegilliland/.pyenv/versions/3.9.16/bin/python
• 3.9.16: /Users/blakegilliland/.pyenv/versions/3.9.16/bin/python3.9
• 3.9.6: /usr/bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.9.16',
 'os_name': 'posix',
 'platform_machine': 'arm64',
 'platform_python_implementation': 'CPython',
 'platform_release': '22.4.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 22.4.0: Mon Mar  6 21:00:41 PST '
                     '2023; root:xnu-8796.101.5~3/RELEASE_ARM64_T8103',
 'python_full_version': '3.9.16',
 'python_version': '3.9',
 'sys_platform': 'darwin'}

System environment variables:

• LDFLAGS
• MANPATH
• TERM_PROGRAM
• PYENV_ROOT
• TERM
• SHELL
• CPPFLAGS
• HOMEBREW_REPOSITORY
• TMPDIR
• RPT_PASSWORD
• TERM_PROGRAM_VERSION
• AWS_SESSION_TOKEN
• ZOHO_TOKEN
• ZOHO_LOGIN
• TERM_SESSION_ID
• PYENV_VERSION
• ZSH
• USER
• COMMAND_MODE
• SSH_AUTH_SOCK
• PYENV_DIR
• __CF_USER_TEXT_ENCODING
• ZOHO_URL
• PAGER
• RPT_HOST
• LSCOLORS
• PATH
• __CFBundleIdentifier
• PWD
• AWS_SECRET_ACCESS_KEY
• LANG
• RPT_USERNAME
• ITERM_PROFILE
• PYENV_HOOK_PATH
• XPC_FLAGS
• XPC_SERVICE_NAME
• AWS_ACCESS_KEY_ID
• SHLVL
• HOME
• COLORFGBG
• LC_TERMINAL_VERSION
• HOMEBREW_PREFIX
• ITERM_SESSION_ID
• LESS
• LOGNAME
• PKG_CONFIG_PATH
• INFOPATH
• HOMEBREW_CELLAR
• LC_TERMINAL
• EGNYTE_ACCESS_TOKEN
• COLORTERM
• PIP_DISABLE_PIP_VERSION_CHECK
• PIP_PYTHON_PATH
• PYTHONDONTWRITEBYTECODE
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /Users/blakegilliland/.pyenv/versions/3.9.16/bin:/opt/homebrew/Cellar/pyenv/2.3.17/libexec:/opt/homebrew/Cellar/pyenv/2.3.17/plugins/python-build/bin:/Users/blakegilliland/.pyenv/shims:/Users/blakegilliland/.pyenv/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/Users/blakegilliland/.pyenv/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/opt/openssl@1.1/bin:/Users/blakegilliland/.pyenv/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin
• SHELL: /bin/zsh
• LANG: en_US.UTF-8
• PWD: /Users/blakegilliland/path/to/project

---

Contents of Pipfile ('/Users/blakegilliland/path/to/project/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]

[dev-packages]

[requires]
python_version = "3.9"

Contents of Pipfile.lock ('/Users/blakegilliland/path/to/project/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "a36a5392bb1e8bbc06bfaa0761e52593cf2d83b486696bf54667ba8da616c839"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.9"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {},
    "develop": {}
}

[HunterDG]

I believe my issue may be related, so commenting here instead of new issue:

I'm unable to pipenv install --python 2.7 on a fresh Ventura MBp . Though it correctly creates a 2.7.18 pyenv, seems it tries to use system (3.9) virtualenv?

brew install pyenv
brew install pipx
brew install pipenv
#or
sudo -H pip3 install -U pipenv
#or
pipx install pipenv
pipenv install --python 2.7

results in:

test@MacBook-Pro test % pipenv install --python 2.7
Warning: Python 2.7 was not found on your system...
Would you like us to install CPython 2.7.18 with Pyenv? [Y/n]: y
Installing CPython 2.7.18 with /opt/homebrew/bin/pyenv (this may take a few minutes)...
✔ Success!
⠦ Installing python...python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
patching file configure
patching file configure.ac
patching file setup.py
patching file 'Mac/Tools/pythonw.c'
patching file setup.py
patching file 'Doc/library/ctypes.rst'
patching file 'Lib/test/test_str.py'
patching file 'Lib/test/test_unicode.py'
patching file 'Modules/_ctypes/_ctypes.c'
patching file 'Modules/_ctypes/callproc.c'
patching file 'Modules/_ctypes/ctypes.h'
patching file 'Modules/_ctypes/callproc.c'
patching file setup.py
patching file 'Mac/Modules/qt/setup.py'
patching file setup.py
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

Creating a virtualenv for this project...
Pipfile: /Users/test/Pipfile
Using /Users/test/.pyenv/versions/2.7.18/bin/python2.7 (2.7.18) to create virtualenv...
⠙ Creating virtual environment...RuntimeError: failed to query /Users/test/.pyenv/versions/2.7.18/bin/python2.7 with code 1 err: '  File "/Library/Python/3.9/site-packages/virtualenv/discovery/py_info.py", line 152\n    os.path.join(base_dir, exe) for exe in (f"python{major}", f"python{major}.{minor}")\n                                                           ^\nSyntaxError: invalid syntax\n'

✘ Failed creating virtual environment
[pipenv.exceptions.VirtualenvCreationException]: 
Failed to create virtual environment.
test@MacBook-Pro test % 

I was finally successful with this workaround:

brew install pyenv
pyenv install 2.7
pyenv shell
pip install --upgrade pip
pip install pipenv
pipenv install  --python 2.7

Further, after above + pyenv global 2.7.18 system, pipenv install --python 2.7 appeared to work correctly - but doesn't this defeat the purpose of pipenv being able to install the required pyenv directly if it doesn't exist?

[matteius]

@HunterDG modern pipenv required python 3.7 or above.

@bgilliland Could you try this branch and see if it resolves your issue? #5686

[HunterDG]

@HunterDG modern pipenv required python 3.7 or above.

to clarify: pipenv install --python 2.7 is not supported in python versions >= 3.7?

[matteius]

I did some major rework to pythonfinder in the release cut an hour ago pipenv==2023.5.19 if anyone could spare checking against this issue, I would expect some improvements.

[matteius]

to clarify: pipenv install --python 2.7 is not supported in python versions >= 3.7?

@HunterDG Correct -- we vendor in the latest pip and that also dropped support for python 3.6 a while back so neither code path could run on < 3.7 for latest versions.

[HunterDG]

Correct

👍🏻 May be worth erroring with specific explanation on that command. Maybe not. Regardless, thanks for the info!

[matteius]

@HunterDG Sounds good, there is another recent ticket tracking that request: #5685

[bgilliland]

Thanks for the reply @matteius, and sorry for taking so long to reply myself. Is the branch you suggested I try now on main ? Looks like it got merged in as pythonfinder-fix-pyenv doesn't exist anymore. Should I just update pipenv in that case?

[matteius]

@bgilliland everything on main was released 2023.6.2 so you can try that from pypi -- nothing significant has been merged to main since then.

[bgilliland]

@matteius I just installed the updated version and retested the issue I had. pipenv installed python using the correct version, 3.9.16 denoted by my pyenv global setting, but my Pipfile has python_version = "3.10". I don't think this is the expected behavior, correct?