tox 3.14.2 uses wrong Python interpreters

[hynek]

When trying to run a tox env using Python 3.8, it actually uses 3.7 (concretely the system interpreter).

I only noticed it because I used hashes in my lockfiles and my builds started failing due to a lack of a hash to dependency that is need on 3.7 but not on 3.8.

Downgrading to homebrew's 3.14.1 fixes it.

tox -rvv:

using tox.ini: /Users/hynek/Work/rdap-svc/tox.ini (pid 49542)
  removing /Users/hynek/Work/rdap-svc/.tox/log
using tox-3.14.2 from /Users/hynek/.local/pipx/venvs/tox/lib/python3.7/site-packages/tox/__init__.py (pid 49542)
GLOB start: packaging
GLOB sdist-make: /Users/hynek/Work/rdap-svc/setup.py
  removing /Users/hynek/Work/rdap-svc/.tox/dist
[49543] /Users/hynek/Work/rdap-svc$ /Users/hynek/.local/pipx/venvs/tox/bin/python setup.py sdist --formats=zip --dist-dir .tox/dist >.tox/log/GLOB-0.log
running sdist
running egg_info
writing src/rdap_svc.egg-info/PKG-INFO
writing dependency_links to src/rdap_svc.egg-info/dependency_links.txt
writing top-level names to src/rdap_svc.egg-info/top_level.txt
reading manifest file 'src/rdap_svc.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching '*.txt'
warning: no previously-included files found matching '*.cfg'
warning: no previously-included files found matching '*.rst'
warning: no previously-included files found matching '*.ini'
warning: no previously-included files found matching '*.yml'
warning: no previously-included files found matching '*.yaml'
warning: no previously-included files found matching '.coveragerc'
warning: no previously-included files found matching 'Makefile'
warning: no previously-included files found matching 'conftest.py'
warning: no previously-included files found matching '.envrc'
warning: no previously-included files found matching 'Dockerfile'
warning: no previously-included files found matching 'test-queries.http'
warning: no previously-included files found matching '.dockerignore'
warning: no previously-included files found matching 'Pipfile*'
warning: no previously-included files found matching '*.toml'
warning: no previously-included files found matching '*.paw'
warning: no previously-included files found matching '*.pot'
no previously-included directories found matching 'tests'
no previously-included directories found matching 'deploy'
no previously-included directories found matching 'requirements'
warning: no files found matching '*.md' under directory 'src'
writing manifest file 'src/rdap_svc.egg-info/SOURCES.txt'
running check
warning: check: missing required meta-data: url

warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied

creating rdap-svc-0
creating rdap-svc-0/src
creating rdap-svc-0/src/rdap_svc
creating rdap-svc-0/src/rdap_svc.egg-info
creating rdap-svc-0/src/rdap_svc/templates
copying files to rdap-svc-0...
copying MANIFEST.in -> rdap-svc-0
copying setup.py -> rdap-svc-0
copying src/rdap_svc/__init__.py -> rdap-svc-0/src/rdap_svc
[... MANY LINES... ]
copying src/rdap_svc.egg-info/PKG-INFO -> rdap-svc-0/src/rdap_svc.egg-info
copying src/rdap_svc.egg-info/SOURCES.txt -> rdap-svc-0/src/rdap_svc.egg-info
copying src/rdap_svc.egg-info/dependency_links.txt -> rdap-svc-0/src/rdap_svc.egg-info
copying src/rdap_svc.egg-info/not-zip-safe -> rdap-svc-0/src/rdap_svc.egg-info
copying src/rdap_svc.egg-info/top_level.txt -> rdap-svc-0/src/rdap_svc.egg-info
copying src/rdap_svc/templates/help.html -> rdap-svc-0/src/rdap_svc/templates
Writing rdap-svc-0/setup.cfg
creating '.tox/dist/rdap-svc-0.zip' and adding 'rdap-svc-0' to it
adding 'rdap-svc-0'
adding 'rdap-svc-0/src'
adding 'rdap-svc-0/PKG-INFO'
adding 'rdap-svc-0/MANIFEST.in'
adding 'rdap-svc-0/setup.py'
adding 'rdap-svc-0/setup.cfg'
adding 'rdap-svc-0/src/rdap_svc.egg-info'
adding 'rdap-svc-0/src/rdap_svc'
adding 'rdap-svc-0/src/rdap_svc.egg-info/PKG-INFO'
adding 'rdap-svc-0/src/rdap_svc.egg-info/not-zip-safe'
adding 'rdap-svc-0/src/rdap_svc.egg-info/SOURCES.txt'
adding 'rdap-svc-0/src/rdap_svc.egg-info/top_level.txt'
adding 'rdap-svc-0/src/rdap_svc.egg-info/dependency_links.txt'
[... MANY LINES ...]
removing 'rdap-svc-0' (and everything under it)

GLOB finish: packaging  after 0.43 seconds
copying new sdistfile to '/Users/hynek/.tox/distshare/rdap-svc-0.zip'
package .tmp/package/1/rdap-svc-0.zip links to dist/rdap-svc-0.zip (/Users/hynek/Work/rdap-svc/.tox)
mypy start: getenv /Users/hynek/Work/rdap-svc/.tox/mypy
mypy cannot reuse: -r flag
mypy recreate: /Users/hynek/Work/rdap-svc/.tox/mypy
python3.8 (/Users/hynek/.local/venvs/rdap-svc/bin/python3.8) is {'executable': '/Users/hynek/.local/pipx/venvs/tox/bin/python', 'name': 'python', 'version_info': [3, 8, 0, 'final', 0], 'version': '3.8.0 (v3.8.0:fa919fdf25, Oct 14 2019, 10:23:27) \n[Clang 6.0 (clang-600.0.57)]', 'is_64': True, 'sysplatform': 'darwin'}
mypy uses /Users/hynek/.local/pipx/venvs/tox/bin/python
  removing /Users/hynek/Work/rdap-svc/.tox/mypy
setting PATH=/Users/hynek/Work/rdap-svc/.tox/mypy/bin:/Users/hynek/.local/venvs/rdap-svc/bin:/Users/hynek/.pyenv/shims:/Users/hynek/.local/bin:/Users/hynek/Library/Python/3.7/bin:/Users/hynek/.poetry/bin:/Users/hynek/.gem/ruby/2.5.0/bin/:/Users/hynek/.gem/ruby/2.6.0/bin/:/usr/local/opt/openssl@1.1/bin:/Users/hynek/Work/go/bin:/Applications/SQLAnywhere17/System/bin64:/usr/local/opt/perl/bin:/Users/hynek/.cargo/bin:/usr/local/opt/gettext/bin:/opt/X11/bin:/Applications/VMware Fusion.app/Contents/Public:/usr/local/MacGPG2/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
[49551] /Users/hynek/Work/rdap-svc/.tox$ /Users/hynek/.local/pipx/venvs/tox/bin/python -m virtualenv --no-download --python /Users/hynek/.local/pipx/venvs/tox/bin/python mypy
Already using interpreter /Users/hynek/.local/pipx/venvs/tox/bin/python
Using base prefix '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7'
New python executable in /Users/hynek/Work/rdap-svc/.tox/mypy/bin/python
Installing setuptools, pip, wheel...
done.
mypy installdeps: -rrequirements/main.txt, -rrequirements/dev.txt
setting PATH=/Users/hynek/Work/rdap-svc/.tox/mypy/bin:/Users/hynek/.local/venvs/rdap-svc/bin:/Users/hynek/.pyenv/shims:/Users/hynek/.local/bin:/Users/hynek/Library/Python/3.7/bin:/Users/hynek/.poetry/bin:/Users/hynek/.gem/ruby/2.5.0/bin/:/Users/hynek/.gem/ruby/2.6.0/bin/:/usr/local/opt/openssl@1.1/bin:/Users/hynek/Work/go/bin:/Applications/SQLAnywhere17/System/bin64:/usr/local/opt/perl/bin:/Users/hynek/.cargo/bin:/usr/local/opt/gettext/bin:/opt/X11/bin:/Applications/VMware Fusion.app/Contents/Public:/usr/local/MacGPG2/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
[49560] /Users/hynek/Work/rdap-svc$ /Users/hynek/Work/rdap-svc/.tox/mypy/bin/python -m pip install -rrequirements/main.txt -rrequirements/dev.txt
Looking in indexes: https://pypi.vm.ag/root/vrmd/+simple/
Collecting attrs==19.3.0
  Using cached https://pypi.vm.ag/root/pypi/%2Bf/08a/96c641c3a74e4/attrs-19.3.0-py2.py3-none-any.whl
[... MANY LINES ...]
Requirement already satisfied: setuptools>=3.0 in ./.tox/mypy/lib/python3.7/site-packages (from gunicorn[setproctitle]==20.0.4->-r requirements/main.txt (line 41)) (41.6.0)
Collecting importlib-metadata>=0.12; python_version < "3.8"
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    importlib-metadata>=0.12; python_version < "3.8" from https://pypi.vm.ag/root/pypi/%2Bf/d95/141fbfa7ef2ec/importlib_metadata-1.3.0-py2.py3-none-any.whl#sha256=d95141fbfa7ef2ec65cfd945e2af7e5a6ddbd7c8d9a25e66ff3be8e3daf9f60f (from pluggy==0.13.1->-r requirements/dev.txt (line 135))
ERROR: invocation failed (exit code 1)
ERROR: could not install deps [-rrequirements/main.txt, -rrequirements/dev.txt]; v = InvocationError('/Users/hynek/Work/rdap-svc/.tox/mypy/bin/python -m pip install -rrequirements/main.txt -rrequirements/dev.txt', 1)
mypy finish: getenv /Users/hynek/Work/rdap-svc/.tox/mypy after 32.84 seconds
__________________________________________________________________________ summary ___________________________________________________________________________
ERROR:   mypy: could not install deps [-rrequirements/main.txt, -rrequirements/dev.txt]; v = InvocationError('/Users/hynek/Work/rdap-svc/.tox/mypy/bin/python -m pip install -rrequirements/main.txt -rrequirements/dev.txt', 1)
cleanup /Users/hynek/Work/rdap-svc/.tox/.tmp/package/1/rdap-svc-0.zip

Please note:

Using base prefix '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7'

which leads to:

$ /Users/hynek/.local/pipx/venvs/tox/bin/python
Python 3.7.3 (default, Nov 15 2019, 04:04:52)
[Clang 11.0.0 (clang-1100.0.33.16)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

Please also note that Python 3.8 is the Python interpreter in the active virtualenv (but tox is not ran out of that virtualenv. It uses the Catalina system 3.7 installed using pipx). IOW it uses the tox interpreter, so I suspect a connection here (homebrew's tox also uses 3.7, just not the system's).

Could this be related to #1462?

[gaborbernat]

@hynek can you validate this by using PyPi tox 3.14.1?

[Julian]

I'm not sure if this is the same issue, a related one or a different one, but I've had on my list to look into that in the last few months tox has started to run pypy toxenvs on the wrong Python too -- specifically, now if you run tox -e pypy, you get pypy3 rather than pypy2.

[asottile]

is there a sample tox.ini that's triggering this, I suspect it's related to an ordering/caching problem and most likely #1462

[hynek]

I don't think the tox.ini is the problem. I can only reproduce it if I use Catalina's system Python it seems.

[gaborbernat]

I don't have access to Catalin yet (our company doesn't allow upgrading to it yet, deah stability) so can't validate fix. When eventually I'll be able to I can try to check out if you have a project that replicates this in a stable fashion.

[gaborbernat]

@hynek I'm 99% this was pypa/virtualenv#1704

[asottile]

@gaborbernat I don't think so, conda isn't involved here and it regressed in a tox changeset, pretty sure the culprit is #1462

[asottile]

yes I can still reproduce:

1. /usr/local/bin/pip3 install --user tox

2. virtualenv venv3 -ppython3.8

3. . venv3/bin/activate

4. this tox.ini

   [tox]
   skipsdist = true
   envlist = py37,py38
   
   [testenv]
   commands =
       python --version

5. /path/to/tox -rvv

(venv3) asottile-mac:y asottile$ /Users/asottile/Library/Python/3.7/bin/tox -rvv
using tox.ini: /private/tmp/y/tox.ini (pid 90128)
  removing /private/tmp/y/.tox/log
using tox-3.14.6 from /Users/asottile/Library/Python/3.7/lib/python/site-packages/tox/__init__.py (pid 90128)
skipping sdist step
py37 start: getenv /private/tmp/y/.tox/py37
py37 cannot reuse: -r flag
py37 recreate: /private/tmp/y/.tox/py37
py37 uses /usr/local/opt/python/bin/python3.7
  removing /private/tmp/y/.tox/py37
setting PATH=/private/tmp/y/.tox/py37/bin:/private/tmp/y/venv3/bin:/usr/local/opt/findutils/libexec/gnubin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
[90130] /private/tmp/y/.tox$ /usr/local/opt/python/bin/python3.7 -m virtualenv --no-download --python /usr/local/opt/python/bin/python3.7 py37
created virtual environment CPython3.7.7.final.0-64 in 298ms
  creator CPython3Posix(dest=/private/tmp/y/.tox/py37, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/Users/asottile/Library/Application Support/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
py37 finish: getenv /private/tmp/y/.tox/py37 after 0.72 seconds
py37 start: finishvenv 
write config to /private/tmp/y/.tox/py37/.tox-config1 as '72f04ae20581db472afbb800b418eeabc5a0b2df5b89e9fb162567cb361519c9 /usr/local/opt/python/bin/python3.7\n3.14.6 0 0 0'
py37 finish: finishvenv  after 0.00 seconds
py37 start: envreport 
setting PATH=/private/tmp/y/.tox/py37/bin:/private/tmp/y/venv3/bin:/usr/local/opt/findutils/libexec/gnubin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
[90132] /private/tmp/y$ /private/tmp/y/.tox/py37/bin/python -m pip freeze >.tox/py37/log/py37-0.log
py37 finish: envreport  after 0.60 seconds
py37 installed: 
py37 start: run-test-pre 
py37 run-test-pre: PYTHONHASHSEED='3851750361'
py37 finish: run-test-pre  after 0.00 seconds
py37 start: run-test 
py37 run-test: commands[0] | python --version
setting PATH=/private/tmp/y/.tox/py37/bin:/private/tmp/y/venv3/bin:/usr/local/opt/findutils/libexec/gnubin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
[90133] /private/tmp/y$ /private/tmp/y/.tox/py37/bin/python --version
Python 3.7.7
py37 finish: run-test  after 0.01 seconds
py37 start: run-test-post 
py37 finish: run-test-post  after 0.00 seconds
py38 start: getenv /private/tmp/y/.tox/py38
py38 cannot reuse: -r flag
py38 recreate: /private/tmp/y/.tox/py38
python3.8 (/private/tmp/y/venv3/bin/python3.8) is {'executable': '/usr/local/Cellar/python/3.7.7/bin/python3.7', 'name': 'python', 'version_info': [3, 8, 0, 'final', 0], 'version': '3.8.0 (default, Jan 13 2020, 10:45:48) \n[Clang 10.0.1 (clang-1001.0.46.4)]', 'is_64': True, 'sysplatform': 'darwin'}
py38 uses /usr/local/Cellar/python/3.7.7/bin/python3.7
  removing /private/tmp/y/.tox/py38
setting PATH=/private/tmp/y/.tox/py38/bin:/private/tmp/y/venv3/bin:/usr/local/opt/findutils/libexec/gnubin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
[90136] /private/tmp/y/.tox$ /usr/local/opt/python/bin/python3.7 -m virtualenv --no-download --python /usr/local/Cellar/python/3.7.7/bin/python3.7 py38
created virtual environment CPython3.7.7.final.0-64 in 295ms
  creator CPython3Posix(dest=/private/tmp/y/.tox/py38, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/Users/asottile/Library/Application Support/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
py38 finish: getenv /private/tmp/y/.tox/py38 after 0.69 seconds
py38 start: finishvenv 
write config to /private/tmp/y/.tox/py38/.tox-config1 as '72f04ae20581db472afbb800b418eeabc5a0b2df5b89e9fb162567cb361519c9 /usr/local/Cellar/python/3.7.7/bin/python3.7\n3.14.6 0 0 0'
py38 finish: finishvenv  after 0.00 seconds
py38 start: envreport 
setting PATH=/private/tmp/y/.tox/py38/bin:/private/tmp/y/venv3/bin:/usr/local/opt/findutils/libexec/gnubin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
[90138] /private/tmp/y$ /private/tmp/y/.tox/py38/bin/python -m pip freeze >.tox/py38/log/py38-0.log
py38 finish: envreport  after 0.59 seconds
py38 installed: 
py38 start: run-test-pre 
py38 run-test-pre: PYTHONHASHSEED='3851750361'
py38 finish: run-test-pre  after 0.00 seconds
py38 start: run-test 
py38 run-test: commands[0] | python --version
setting PATH=/private/tmp/y/.tox/py38/bin:/private/tmp/y/venv3/bin:/usr/local/opt/findutils/libexec/gnubin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
[90139] /private/tmp/y$ /private/tmp/y/.tox/py38/bin/python --version
Python 3.7.7
py38 finish: run-test  after 0.01 seconds
py38 start: run-test-post 
py38 finish: run-test-post  after 0.00 seconds
___________________________________ summary ____________________________________
  py37: commands succeeded
  py38: commands succeeded
  congratulations :)