Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pipenv installs into .venv even though PIPENV_VENV_IN_PROJECT is not set #2763

Closed
thernstig opened this issue Aug 20, 2018 · 17 comments · Fixed by #5026
Closed

pipenv installs into .venv even though PIPENV_VENV_IN_PROJECT is not set #2763

thernstig opened this issue Aug 20, 2018 · 17 comments · Fixed by #5026
Labels
PR: awaiting-review The PR related to this issue is awaiting review by a maintainer. Type: Bug 🐛 This issue is a bug.

Comments

@thernstig
Copy link

thernstig commented Aug 20, 2018

Issue description

If a folder exists called .venv, pipenv will automatically install into it even though PIPENV_VENV_IN_PROJECT is not set.

Expected result

I expected it to install into the global virtual environment location (usually /home/moo/.local/share/virtualenvs/)

Actual result

It installed into ./.venv/

Extra information

It is worth to note what I am trying to achieve.

I am running Windows and VS Code installed in Windows. But I am also running WSL. What I am trying to do is to use WSL (bash) as my terminal in VS Code. There are few things to consider here.

  1. I have installed python both in Windows and WSL/bash (Ubuntu).
  2. I have set PIPENV_VENV_IN_PROJECT in Windows, but not in WSL.
  3. The VS Code extension is configured like "python.pythonPath": "${workspaceFolder}/.venv",. For the VS Code extension to work, I need to execute pipenv install in Windows Powershell or cmd (i.e. in Windows). This is since VS Code is installed in Windows.
  4. However, when I open the integrated terminal in VS Code I want to use WSL. As such, it cannot use .venv since everything installed there is Windows-specific. So I try to do pipenv install in the integrated terminal (running WSL/bash) in the hopes that it will install to the global venv location. This way, I could properly use both VS Code with the python extension, and at the same time use the integrated terminal to run things there as well.
Steps to replicate
14:44:36 DESKTOP-41D0GKF ~/test$ ll
total 0
drwxrwxrwx 1 moo moo 512 Aug 20 14:44 ./
drwxr-xr-x 1 moo moo 512 Aug 20 14:43 ../
14:44:39 DESKTOP-41D0GKF ~/test$ pipenv install requests
Creating a virtualenv for this project...
Pipfile: /home/moo/test/Pipfile
Using /usr/bin/python3 (3.5.2) to create virtualenv...
⠋Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/moo/.local/share/virtualenvs/test-O3XhBtkW/bin/python3
Also creating executable in /home/moo/.local/share/virtualenvs/test-O3XhBtkW/bin/python
Installing setuptools, pip, wheel...done.
Setting project for test-O3XhBtkW to /home/moo/test

Virtualenv location: /home/moo/.local/share/virtualenvs/test-O3XhBtkW
Creating a Pipfile for this project...                                                                                                                                                                                                      Installing requests...
Collecting requests
  Using cached https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests)
  Using cached https://files.pythonhosted.org/packages/16/1f/50d729c104b21c1042aa51560da6141d1cab476ba7015d92b2111c8db841/certifi-2018.8.13-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.8,>=2.5 (from requests)
  Using cached https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl
Collecting urllib3<1.24,>=1.21.1 (from requests)
  Using cached https://files.pythonhosted.org/packages/bd/c9/6fdd990019071a4a32a5e7cb78a1d92c53851ef4f56f62a3486e6a7d8ffb/urllib3-1.23-py2.py3-none-any.whl
Installing collected packages: certifi, chardet, idna, urllib3, requests
Successfully installed certifi-2018.8.13 chardet-3.0.4 idna-2.7 requests-2.19.1 urllib3-1.23

Adding requests to Pipfile's [packages]...
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Updated Pipfile.lock (0b4483)!
Installing dependencies from Pipfile.lock (0b4483)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 5/5 — 00:00:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
14:45:07 DESKTOP-41D0GKF ~/test$
14:48:39 DESKTOP-41D0GKF ~/test$
14:48:39 DESKTOP-41D0GKF ~/test$ ll
total 4
drwxrwxrwx 1 moo moo  512 Aug 20 14:45 ./
drwxr-xr-x 1 moo moo  512 Aug 20 14:43 ../
-rw-rw-rw- 1 moo moo  153 Aug 20 14:45 Pipfile
-rw-r--r-- 1 moo moo 2106 Aug 20 14:45 Pipfile.lock
14:48:40 DESKTOP-41D0GKF ~/test$ pipenv --venv
/home/moo/.local/share/virtualenvs/test-O3XhBtkW
14:48:45 DESKTOP-41D0GKF ~/test$ pipenv --rm
Removing virtualenv (/home/moo/.local/share/virtualenvs/test-O3XhBtkW)...
14:48:49 DESKTOP-41D0GKF ~/test$ mkdir .venv
14:48:51 DESKTOP-41D0GKF ~/test$ pipenv install requests
Creating a virtualenv for this project...
Pipfile: /home/moo/test/Pipfile
Using /usr/bin/python3.5m (3.5.2) to create virtualenv...
⠋Running virtualenv with interpreter /usr/bin/python3.5m
Using base prefix '/usr'
New python executable in /home/moo/test/.venv/bin/python3.5m
Also creating executable in /home/moo/test/.venv/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /home/moo/test/.venv                                                                                                                                                                                                Installing requests...
Collecting requests
  Using cached https://files.pythonhosted.org/packages/65/47/7e02164a2a3db50ed6d8a6ab1d6d60b69c4c3fdf57a284257925dfc12bda/requests-2.19.1-py2.py3-none-any.whl
Collecting idna<2.8,>=2.5 (from requests)
  Using cached https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests)
  Using cached https://files.pythonhosted.org/packages/16/1f/50d729c104b21c1042aa51560da6141d1cab476ba7015d92b2111c8db841/certifi-2018.8.13-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting urllib3<1.24,>=1.21.1 (from requests)
  Using cached https://files.pythonhosted.org/packages/bd/c9/6fdd990019071a4a32a5e7cb78a1d92c53851ef4f56f62a3486e6a7d8ffb/urllib3-1.23-py2.py3-none-any.whl
Installing collected packages: idna, certifi, chardet, urllib3, requests
Successfully installed certifi-2018.8.13 chardet-3.0.4 idna-2.7 requests-2.19.1 urllib3-1.23

Adding requests to Pipfile's [packages]...
Installing dependencies from Pipfile.lock (0b4483)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 5/5 — 00:00:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
14:49:10 DESKTOP-41D0GKF ~/test$ pipenv install requests^C
14:50:24 DESKTOP-41D0GKF ~/test$ ll
total 4
drwxrwxrwx 1 moo moo  512 Aug 20 14:48 ./
drwxr-xr-x 1 moo moo  512 Aug 20 14:43 ../
-rw-rw-rw- 1 moo moo  153 Aug 20 14:45 Pipfile
-rw-r--r-- 1 moo moo 2106 Aug 20 14:45 Pipfile.lock
drwxrwxrwx 1 moo moo  512 Aug 20 14:49 .venv/
14:50:25 DESKTOP-41D0GKF ~/test$ pipenv --venv
/home/moo/test/.venv
14:50:29 DESKTOP-41D0GKF ~/test$

$ pipenv --support

Pipenv version: '2018.7.1'

Pipenv location: '/usr/local/lib/python3.5/dist-packages/pipenv'

Python location: '/usr/bin/python3'

Other Python installations in PATH:

  • 2.7: /usr/bin/python2.7

  • 2.7: /usr/bin/python2.7

  • 3.5: /usr/bin/python3.5m

  • 3.5: /usr/bin/python3.5

  • 3.6: /usr/bin/python3.6m

  • 3.6: /usr/bin/python3.6

  • 2.7.12: /usr/bin/python

  • 2.7.12: /usr/bin/python2

  • 3.6.6: /usr/local/bin/python3

  • 3.5.2: /usr/bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.5.2',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.4.0-17134-Microsoft',
 'platform_system': 'Linux',
 'platform_version': '#137-Microsoft Thu Jun 14 18:46:00 PST 2018',
 'python_full_version': '3.5.2',
 'python_version': '3.5',
 'sys_platform': 'linux'}

System environment variables:

  • XDG_DATA_DIRS
  • LESSCLOSE
  • USER
  • LANG
  • TERM
  • _
  • LOGNAME
  • SHLVL
  • OLDPWD
  • LS_COLORS
  • LESSOPEN
  • PIP_PYTHON_PATH
  • SHELL
  • NAME
  • PS1
  • HOSTTYPE
  • PWD
  • PYTHONDONTWRITEBYTECODE
  • PATH
  • HOME

Pipenv–specific environment variables:

Debug–specific environment variables:

  • PATH: /home/moo/bin:/home/moo/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/Program Files/Python36/Scripts:/mnt/c/Program Files/Python36:/mnt/c/Program Files/Docker/Docker/resources/bin:/mnt/c/Windows/System32:/mnt/c/Windows:/mnt/c/Windows/System32/wbem:/mnt/c/Windows/System32/WindowsPowerShell/v1.0:/mnt/c/Program Files/ctags58:/mnt/c/Program Files (x86)/Windows Kits/8.1/Windows Performance Toolkit:/mnt/c/Program Files/Microsoft VS Code/bin:/mnt/c/Program Files/Git/cmd:/mnt/c/Windows/System32/OpenSSH:/mnt/c/Users/moo/AppData/Local/Microsoft/WindowsApps:/snap/bin
  • SHELL: /bin/bash
  • LANG: en_US.UTF-8
  • PWD: /home/moo/test

Contents of Pipfile ('/home/moo/test/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
requests = "*"

[dev-packages]

[requires]
python_version = "3.5"

Contents of Pipfile.lock ('/home/moo/test/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "50b8935d42537b1d392d8fe9a921302d58566b8b364c66af879714c0220b4483"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.5"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "certifi": {
            "hashes": [
                "sha256:4c1d68a1408dd090d2f3a869aa94c3947cc1d967821d1ed303208c9f41f0f2f4",
                "sha256:b6e8b28b2b7e771a41ecdd12d4d43262ecab52adebbafa42c77d6b57fb6ad3a4"
            ],
            "version": "==2018.8.13"
        },
        "chardet": {
            "hashes": [
                "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae",
                "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691"
            ],
            "version": "==3.0.4"
        },
        "idna": {
            "hashes": [
                "sha256:156a6814fb5ac1fc6850fb002e0852d56c0c8d2531923a51032d1b70760e186e",
                "sha256:684a38a6f903c1d71d6d5fac066b58d7768af4de2b832e426ec79c30daa94a16"
            ],
            "version": "==2.7"
        },
        "requests": {
            "hashes": [
                "sha256:63b52e3c866428a224f97cab011de738c36aec0185aa91cfacd418b5d58911d1",
                "sha256:ec22d826a36ed72a7358ff3fe56cbd4ba69dd7a6718ffd450ff0e9df7a47ce6a"
            ],
            "index": "pypi",
            "version": "==2.19.1"
        },
        "urllib3": {
            "hashes": [
                "sha256:a68ac5e15e76e7e5dd2b8f94007233e01effe3e50e8daddf69acfd81cb686baf",
                "sha256:b5725a0bd4ba422ab0e66e89e030c806576753ea3ee08554382c14e685d117b5"
            ],
            "markers": "python_version >= '2.6' and python_version != '3.0.*' and python_version < '4' and python_version != '3.3.*' and python_version != '3.2.*' and python_version != '3.1.*'",
            "version": "==1.23"
        }
    },
    "develop": {}
}
@techalchemy
Copy link
Member

Had to know this would cause problems somehow!

@techalchemy
Copy link
Member

I don’t even think there is a way to prevent this behavior. Sorry for the issue!

@thernstig
Copy link
Author

@techalchemy That is unfortunate, considering my usage scenario.

@joshfriend
Copy link
Contributor

This behavior was explicitly added in #1492

@foozmeat
Copy link

foozmeat commented Jan 29, 2019

This behavior has bitten me today.

I mount my django app into a vagrant instance and set up the virtual environment into .venv but when I try to create a development venv on my host mac it picks up the .venv created in the VM and dies. Setting PIPENV_IGNORE_VIRTUALENVS doesn't change the behavior.

Being able to set the venv location explicitly via an env var would solve this problem for me.

@th3hamm0r
Copy link

th3hamm0r commented Apr 8, 2019

@foozmeat we've the same setup and problems at the moment...
@techalchemy It would be nice if we could disable this auto-detection with an env variable similar to PIPENV_IGNORE_VIRTUALENVS or as @foozmeat suggested, with a variable overriding the location explicitly.
The later one would also give us an additional way of setting the venv-location (related to #1049).

@rikroe
Copy link

rikroe commented Oct 29, 2019

Would it be possible to use different folder names and then OS-depended checks? I.e. folder .venv for Linux/WSL and venv (without a dot) for Windows?
This way both OS could have their respective executables but share the same environment?

@abely
Copy link

abely commented Nov 8, 2019

The code in #1492 looks like this:

PIPENV_VENV_IN_PROJECT = bool(os.environ.get('PIPENV_VENV_IN_PROJECT')) or os.path.isdir('.venv')`

So a workaround could be setting PIPENV_VENV_IN_PROJECT to false? (didn't test it myself)

@thernstig
Copy link
Author

@abely That is the problem here, not having it set (or setting it to false) will go to the or branch and still use the .venv dir.

@akzincsystems
Copy link

Well, it looks like where to put the environment is still biting after many years of debate. Came across this one after failing to the flag to work.

So now it's totally broken:

  • set the flag - it's ignored unless you also create a project .venv directory
  • BUT - pipenv will respect a local venv in all its other command variants!

Node/Javascript's npm and PHPs composer dependency management simply don't have this problem. Devs expect the damn runtime dependencies to be in their local directory. They .gitignore them for the repos. When they build for deployment the dependency lock file takes care of things and installs everything that is required, where it is expected to be. Down the build chain all the tools know exactly where to find things. Really, really simple

How about having an additional flag for the base command, e.g.
pipenv --three --local
--local once and for all determines where to put the venv directory, i.e in the project directory. End of pain, and under dev control. Up to them to sort out anything downstream of it.

The world is not where it was 7 or 8 years ago. We are deploying to docker containers and lambda, not cold metal anymore.

@jsteenri
Copy link

Upvoting @akzincsystems comment, a first-class command-line "put it here" directive like '--local' would be ideal.

Note that with (at least) version 2020.11.15, the presence of Pipfile{,.lock} in a(ny) directory above the current (e.g. project) one supersedes the desired local creation one expects with PIPENV_VENV_IN_PROJECT set. Make sure pipenv can't find remnants of an existing virtual environment anywhere else.

@smac89
Copy link
Contributor

smac89 commented Mar 30, 2022

Bump! Please address this issue. Even when I set it to a falsey value, it still tries to use .venv. The problem stems from the way pipenv checks if the project is using .venv:

pipenv/pipenv/project.py

Lines 243 to 248 in 9a3b3ce

def is_venv_in_project(self):
# type: () -> bool
return self.s.PIPENV_VENV_IN_PROJECT or (
self.project_directory
and os.path.isdir(os.path.join(self.project_directory, ".venv"))
)

However, I was able get this working for me during development inside a docker container. Here are the commands I run:

ln -sf "${PWD}/Pipfile" /tmp/Pipfile
ln -sf "${PWD}/Pipfile.lock" /tmp/Pipfile.lock
PIPENV_PIPFILE=/tmp/Pipfile pipenv install

The true champion is PIPENV_PIPFILE which basically works as a way of telling pipenv where the project directory is. This means that pipenv will not be looking in the current directory which may contain a .venv folder.

@matteius matteius added Type: Bug 🐛 This issue is a bug. Priority: High This item is high priority and should be resolved quickly. triage labels Mar 30, 2022
@matteius
Copy link
Member

matteius commented Mar 30, 2022

@smac89 I don't quite understand the issue with the logical check in is_venv_in_project -- since you don't have PIPENV_VENV_IN_PROJECT it would assume your project must already have a .venv directory its detecting and using that instead of creating the virtualenv outside of the project. Is that really a bug? Also, I assume you are running the newest release?

@matteius
Copy link
Member

Maybe mine works fine because I have a WORKON_HOME environment variable set? For me it appears to use that.

@smac89
Copy link
Contributor

smac89 commented Mar 30, 2022

@matteius the problem is that even if we set PIPENV_VENV_IN_PROJECT to a falsey value such as 0 (as if to say that we don't want to create the virtual env inside the project, pipenv ignores, this flag and still attempts to detect a .venv folder inside the project directory.

The correct solution should be to allow the user to explicitly opt in or out by setting PIPENV_VENV_IN_PROJECT to true or false, therefore only in the absence of PIPENV_VENV_IN_PROJECT should pipenv's current behaviour be acceptable.

Also yes I am using the lastest version

@smac89
Copy link
Contributor

smac89 commented Mar 30, 2022

@matteius If it helps, here is one use-case where this distinction between PIPENV_VENV_IN_PROJECT being unset vs being set to a falsey value is important:

When we do python development, we test everything with docker-compose, and we usually mount our current working directory as a volume inside the container. Now sometimes, that current working directory may contain a .venv folder which contains python binaries from our host machine.
If we try to use this same virtual env inside the docker container, it will fail because the location to that python binary does not exist inside the container.

Now if only there was a way to tell pipenv to ignore this .venv folder, and either install the packages globally within the container, or create it's own virtualenv outside of the project directory? That's where PIPENV_VENV_IN_PROJECT comes in, but it is currently not meeting that requirement, unfortunately.

@matteius
Copy link
Member

Thanks @smac89 can you try out this branch and see if it solves the issue? #5026

@matteius matteius added PR: awaiting-review The PR related to this issue is awaiting review by a maintainer. and removed Priority: High This item is high priority and should be resolved quickly. triage labels Apr 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PR: awaiting-review The PR related to this issue is awaiting review by a maintainer. Type: Bug 🐛 This issue is a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.