hundreds of processes spawned

[timfelle]

After installing python 3.7.4 64 bit, i attempted to start a virtual environment in a folder. However, about a thousand python processes were started and the virtual environment did not get completed.

OS: Windows 10 Home
Code run in Cygwin 64 bit

Failing code:

mkdir test
cd test
virtualenv venv

pip list

Package Version

---

astroid 2.2.0
colorama 0.4.1
isort 4.3.9
lazy-object-proxy 1.3.1
mccabe 0.6.1
pip 19.1.1
pylint 2.3.0
rope 0.14.0
setuptools 40.8.0
six 1.12.0
typed-ast 1.3.1
virtualenv 16.6.1
wrapt 1.11.1

[AndrYast]

I tried to install python, pip and virtualenv for the first time today and faced the same problem.

I've fixed it by commenting 3 lines in
Python\Python37\Lib\site-packages\virtualenv.py
and adding
-p python
when running virtualenv

The lines i commented are 783-785:
if hasattr(sys, "_base_executable"):
print("hasattr(sys, \"_base_executable\") == yes")
return sys._base_executable

[thingselliotprograms]

@AndrYast after activating the environment are you still on 3.7.4? This definitely seems to be an issue specifically with 3.7.4, which was released yesterday (7/8/19).

[AndrYast]

@thingselliotprograms yes, running
venv\Scripts\activate
python --version
gives me
Python 3.7.4

[grundysf]

Experiencing same issue with pipenv hanging since upgrading python yesterday to 3.7.4. Underlying issue related to virtualenv. Downgraded python to 3.7.3 for now.

[gaborbernat]

Ack virtualenv latest is broken because python/cpython#14428 always sets sys._base_executable.

[akittas]

I'm getting the same issue with 3.7.4. I'm using pipenv and getting a [WinError 8] Not enough memory resources are available to process this command. I cannot create any virtualenv. Everything works great in 3.7.3.

[fireundubh]

I'm experiencing this exact issue, too, with 3.7.4.

Rolling thunder of python.exe processes exhausts memory and virtualenv creation ultimately fails.

The issue did not exist with 3.7.2.

[gaborbernat]

Maintainer here. As I said above the contract for some internal variable of CPython changed, and this causes now infinite loop on process creation for both python 3.7 and 3.8.

[fireundubh]

No idea whether this is a good idea, but I was able to create a virtual environment with this quick hack.

# virtualenv.py:783
if hasattr(sys, "_base_executable") and sys.version_info < (3, 7, 4):
    return sys._base_executable

Judging by the comments above that line, looks like you had some fun in an earlier version.

[pfmoore]

Maybe there needs to be an additional check that sys._base_executable != sys.executable. But honestly, I don't know - this seems to be continually changing in patch releases and I don't have the time to keep up with what's going on (it all seems to be related to work to support the "Windows Store" build of Python).

Maybe @zooba could comment or offer suggestions here. We're using undocumented internals, so ultimately the issue is up to us to address, but I don't know of a supported way to get the information we need, so I don't see a solution that won't be subject to potential breakage every time we get a new release of Python :-(

Ultimately, #1377 is probably the only reliable solution here.

[gaborbernat]

I think as long as we add that check we'll be fine.

[YakDriver]

Travis CI jobs running Windows and attempting to create a virtualenv also fail. Travis CI recently moved to 3.7.4. 😞

https://travis-ci.community/t/infinite-loop-of-virtualenv-windows/4139

Possibly helpful information from the Travis builds:

version: v6.2.0 https://github.com/travis-ci/worker/tree/5e5476e01646095f48eec13196fdb3faf8f5cbf7
instance: travis-ci-onion-1803-containers-1542208204-ad01dca (via amqp)
bash version 4.4.19(2)-release
Chocolatey v0.10.11
python3 v3.7.4 [Approved]
python3 has been installed.
Successfully installed pip-19.1.1
Successfully installed virtualenv-16.6.1

$ virtualenv $HOME/venv
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
Running virtualenv with interpreter c:\python37\python.exe
...[repeats 763 more times]...
The command "virtualenv $HOME/venv" failed and exited with 1 during .
Your build has been stopped.
MemoryError

[pfmoore]

I think as long as we add that check we'll be fine.

Won't doing that reintroduce (on 3.7.4) the issues the base_executable check was intended to fix? Admittedly those issues were far less serious, so it's likely still worth doing, but I don't think it's a full solution.

[gaborbernat]

I don't think it causes regression, does it?