tox fails to create py27 virtualenv - installs packages to system site packages

[jaraco]

Attempting to run tox against Python 2.7, I find that all commands are run in the system and not in the environment.

Tox is installed thus:

$ tox --version                                                                                                                                                     
3.14.0 imported from /Users/jaraco/.local/pipx/venvs/tox/lib/python3.8/site-packages/tox/__init__.py
registered plugins:
    tox-pip-version-0.0.5 at /Users/jaraco/.local/pipx/venvs/tox/lib/python3.8/site-packages/tox_pip_version/hooks.py
    tox-venv-0.4.0 at /Users/jaraco/.local/pipx/venvs/tox/lib/python3.8/site-packages/tox_venv/hooks.py

As you can see, I've used pipx to install tox, so it's created in a pyvenv in ~/.local/pipx/tox.

I create this tox.ini:

[tox]
skipsdist=True

[testenv]
commands = python -V
skipinstall = True

I then run tox:

draft $ tox -vv -e py27                                                                                                                                                   
using tox.ini: /Users/jaraco/draft/tox.ini (pid 18734)
using tox-3.14.0 from /Users/jaraco/.local/pipx/venvs/tox/lib/python3.8/site-packages/tox/__init__.py (pid 18734)
skipping sdist step
py27 start: getenv /Users/jaraco/draft/.tox/py27
py27 cannot reuse: no previous config /Users/jaraco/draft/.tox/py27/.tox-config1
py27 create: /Users/jaraco/draft/.tox/py27
python2.7 (/usr/local/bin/python2.7) is {'name': 'python', 'sysplatform': 'darwin', 'version_info': [2, 7, 16, 'final', 0], 'executable': '/usr/local/opt/python@2/bin/python2.7', 'version': '2.7.16 (default, Sep  2 2019, 11:59:44) \n[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)]', 'is_64': True}
py27 uses /usr/local/opt/python@2/bin/python2.7
WARNING: test command found but not installed in testenv
  cmd: /usr/local/bin/python
  env: /Users/jaraco/draft/.tox/py27
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.

DEPRECATION WARNING: this will be an error in tox 4 and above!
py27 finish: getenv /Users/jaraco/draft/.tox/py27 after 0.09 seconds
py27 start: finishvenv 
write config to /Users/jaraco/draft/.tox/py27/.tox-config1 as '9aa6efa7d673a644c96ee57fbb55932974df5d32c590caab2f44c0d4e652324f /usr/local/opt/python@2/bin/python2.7\n3.14.0 0 0 0'
py27 finish: finishvenv  after 0.00 seconds
py27 start: envreport 
setting PATH=/Users/jaraco/draft/.tox/py27/bin:/Users/jaraco/.local/bin:/usr/local/opt/openssl@1.1/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/Library/Frameworks/Python.framework/Versions/3.7/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/Library/Frameworks/Python.framework/Versions/3.5/bin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/Users/jaraco/.local/homebrew/bin:/usr/local/heroku/bin:/Users/jaraco/Dropbox/bin/mac:/opt/cisco/anyconnect/bin:/Users/jaraco/Dropbox/bin/scripts:/Applications/Sublime Merge.app/Contents/SharedSupport/bin:/Applications/Sublime Text.app/Contents/SharedSupport/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/Library/Apple/bin:/Applications/VMware Fusion.app/Contents/Public:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Applications/Wireshark.app/Contents/MacOS:/Users/jaraco/.local/bin
WARNING: test command found but not installed in testenv
  cmd: /usr/local/bin/python
  env: /Users/jaraco/draft/.tox/py27
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.

DEPRECATION WARNING: this will be an error in tox 4 and above!
[18739] /Users/jaraco/draft$ /usr/local/bin/python -m pip freeze >.tox/py27/log/py27-0.log
py27 finish: envreport  after 0.26 seconds
py27 installed: argcomplete==1.10.0,Click==7.0,distro==1.4.0,pip-run==5.3,pipx==0.14.0.0,userpath==1.2.0
py27 start: run-test-pre 
py27 run-test-pre: PYTHONHASHSEED='163692504'
py27 finish: run-test-pre  after 0.00 seconds
py27 start: run-test 
py27 run-test: commands[0] | python -V
setting PATH=/Users/jaraco/draft/.tox/py27/bin:/Users/jaraco/.local/bin:/usr/local/opt/openssl@1.1/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/Library/Frameworks/Python.framework/Versions/3.7/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/Library/Frameworks/Python.framework/Versions/3.5/bin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/Users/jaraco/.local/homebrew/bin:/usr/local/heroku/bin:/Users/jaraco/Dropbox/bin/mac:/opt/cisco/anyconnect/bin:/Users/jaraco/Dropbox/bin/scripts:/Applications/Sublime Merge.app/Contents/SharedSupport/bin:/Applications/Sublime Text.app/Contents/SharedSupport/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/Library/Apple/bin:/Applications/VMware Fusion.app/Contents/Public:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Applications/Wireshark.app/Contents/MacOS:/Users/jaraco/.local/bin
WARNING: test command found but not installed in testenv
  cmd: /usr/local/bin/python
  env: /Users/jaraco/draft/.tox/py27
Maybe you forgot to specify a dependency? See also the whitelist_externals envconfig setting.

DEPRECATION WARNING: this will be an error in tox 4 and above!
[18740] /Users/jaraco/draft$ /usr/local/bin/python -V
Python 3.8.0rc1
py27 finish: run-test  after 0.01 seconds
py27 start: run-test-post 
py27 finish: run-test-post  after 0.00 seconds
________________________________________________________________________________ summary _________________________________________________________________________________
  py27: commands succeeded
  congratulations :)

And the env isn't working:

$ ls .tox/py27/                                                                                                                                                     
log tmp

If I had specified deps in the tox.ini, they would have been installed into my system site packages.

[jaraco]

If I remove tox-venv the behavior goes more or less as I'd expect:

draft $ tox -vvr -e py27                                                                                                                                                  
using tox.ini: /Users/jaraco/draft/tox.ini (pid 18918)
using tox-3.14.0 from /Users/jaraco/.local/pipx/venvs/tox/lib/python3.8/site-packages/tox/__init__.py (pid 18918)
skipping sdist step
py27 start: getenv /Users/jaraco/draft/.tox/py27
py27 cannot reuse: -r flag
py27 create: /Users/jaraco/draft/.tox/py27
python2.7 (/usr/local/bin/python2.7) is {'name': 'python', 'version': '2.7.16 (default, Sep  2 2019, 11:59:44) \n[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)]', 'sysplatform': 'darwin', 'executable': '/usr/local/opt/python@2/bin/python2.7', 'version_info': [2, 7, 16, 'final', 0], 'is_64': True}
py27 uses /usr/local/opt/python@2/bin/python2.7
setting PATH=/Users/jaraco/draft/.tox/py27/bin:/Users/jaraco/.local/bin:/usr/local/opt/openssl@1.1/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/Library/Frameworks/Python.framework/Versions/3.7/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/Library/Frameworks/Python.framework/Versions/3.5/bin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/Users/jaraco/.local/homebrew/bin:/usr/local/heroku/bin:/Users/jaraco/Dropbox/bin/mac:/opt/cisco/anyconnect/bin:/Users/jaraco/Dropbox/bin/scripts:/Applications/Sublime Merge.app/Contents/SharedSupport/bin:/Applications/Sublime Text.app/Contents/SharedSupport/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/Library/Apple/bin:/Applications/VMware Fusion.app/Contents/Public:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Applications/Wireshark.app/Contents/MacOS:/Users/jaraco/.local/bin
[18922] /Users/jaraco/draft/.tox$ /Users/jaraco/.local/pipx/venvs/tox/bin/python -m virtualenv --no-download --python /usr/local/opt/python@2/bin/python2.7 py27
Running virtualenv with interpreter /usr/local/opt/python@2/bin/python2.7
Already using interpreter /usr/local/opt/python@2/bin/python2.7
New python executable in /Users/jaraco/draft/.tox/py27/bin/python2.7
Also creating executable in /Users/jaraco/draft/.tox/py27/bin/python
Installing setuptools, pip, wheel...
done.
py27 finish: getenv /Users/jaraco/draft/.tox/py27 after 2.86 seconds
py27 start: finishvenv 
write config to /Users/jaraco/draft/.tox/py27/.tox-config1 as '9aa6efa7d673a644c96ee57fbb55932974df5d32c590caab2f44c0d4e652324f /usr/local/opt/python@2/bin/python2.7\n3.14.0 0 0 0'
py27 finish: finishvenv  after 0.00 seconds
py27 start: envreport 
setting PATH=/Users/jaraco/draft/.tox/py27/bin:/Users/jaraco/.local/bin:/usr/local/opt/openssl@1.1/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/Library/Frameworks/Python.framework/Versions/3.7/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/Library/Frameworks/Python.framework/Versions/3.5/bin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/Users/jaraco/.local/homebrew/bin:/usr/local/heroku/bin:/Users/jaraco/Dropbox/bin/mac:/opt/cisco/anyconnect/bin:/Users/jaraco/Dropbox/bin/scripts:/Applications/Sublime Merge.app/Contents/SharedSupport/bin:/Applications/Sublime Text.app/Contents/SharedSupport/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/Library/Apple/bin:/Applications/VMware Fusion.app/Contents/Public:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Applications/Wireshark.app/Contents/MacOS:/Users/jaraco/.local/bin
[18930] /Users/jaraco/draft$ /Users/jaraco/draft/.tox/py27/bin/python -m pip freeze >.tox/py27/log/py27-0.log
py27 finish: envreport  after 0.30 seconds
py27 installed: DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
py27 start: run-test-pre 
py27 run-test-pre: PYTHONHASHSEED='2627647495'
py27 finish: run-test-pre  after 0.00 seconds
py27 start: run-test 
py27 run-test: commands[0] | python -V
setting PATH=/Users/jaraco/draft/.tox/py27/bin:/Users/jaraco/.local/bin:/usr/local/opt/openssl@1.1/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/Library/Frameworks/Python.framework/Versions/3.7/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/Library/Frameworks/Python.framework/Versions/3.5/bin:/Library/Frameworks/Python.framework/Versions/3.4/bin:/Users/jaraco/.local/homebrew/bin:/usr/local/heroku/bin:/Users/jaraco/Dropbox/bin/mac:/opt/cisco/anyconnect/bin:/Users/jaraco/Dropbox/bin/scripts:/Applications/Sublime Merge.app/Contents/SharedSupport/bin:/Applications/Sublime Text.app/Contents/SharedSupport/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/Library/Apple/bin:/Applications/VMware Fusion.app/Contents/Public:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Applications/Wireshark.app/Contents/MacOS:/Users/jaraco/.local/bin
[18931] /Users/jaraco/draft$ /Users/jaraco/draft/.tox/py27/bin/python -V
Python 2.7.16
py27 finish: run-test  after 0.01 seconds
py27 start: run-test-post 
py27 finish: run-test-post  after 0.00 seconds
________________________________________________________________________________ summary _________________________________________________________________________________
  py27: commands succeeded
  congratulations :)

[jaraco]

I was able to replicate the issue in an isolated Docker image with this Dockerfile:

FROM jaraco/py37-tox
RUN apt install -y python2.7
RUN python -m venv tox-venv
RUN tox-venv/bin/python -m pip install tox tox-venv tox-pip-version
RUN echo "[tox]\nskipsdist = True\n[testenv]\ncommands = python -V\nskipinstall = True" > tox.ini
RUN tox-venv/bin/tox -e py27

In doing so, I learned that tox-pip-version is implicated also.

[jaraco]

I suggest this issue is a major blocker as the use of tox-pip-version is the only sanctioned workaround for long-standing issues like tox-dev/tox-venv#17.

[jaraco]

I've observed that removing either of tox-venv or tox-pip-version bypasses the error.

[gaborbernat]

I see this as a bug of either tox-venv or tox-pip-version. This will be provided internally with #1400, not to be released though until early next year as for now I'm busy with pypa/virtualenv#1377