Errors if WORKON_HOME is not writable could be better

[OrangeDog]

Pipenv version: 2018.7.1
The first error is OK (the PermissionError), but then there's a lot more.

$ WORKON_HOME=/opt/ pipenv sync
Creating a virtualenv for this project...
Pipfile: /home/me/project/Pipfile
Using /usr/bin/python3.6m (3.6.5) to create virtualenv...
⠋Running virtualenv with interpreter /usr/bin/python3.6m
Using base prefix '/usr'
Traceback (most recent call last):
  File "/home/me/.local/lib/python3.6/site-packages/virtualenv.py", line 2343, in <module>
    main()
  File "/home/me/.local/lib/python3.6/site-packages/virtualenv.py", line 712, in main
    symlink=options.symlink)
  File "/home/me/.local/lib/python3.6/site-packages/virtualenv.py", line 927, in create_environment
    site_packages=site_packages, clear=clear, symlink=symlink))
  File "/home/me/.local/lib/python3.6/site-packages/virtualenv.py", line 1112, in install_python
    mkdir(lib_dir)
  File "/home/me/.local/lib/python3.6/site-packages/virtualenv.py", line 324, in mkdir
    os.makedirs(path)
  File "/usr/lib/python3.6/os.py", line 210, in makedirs
    makedirs(head, mode, exist_ok)
  File "/usr/lib/python3.6/os.py", line 210, in makedirs
    makedirs(head, mode, exist_ok)
  File "/usr/lib/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/opt/clump-D6KMH_ut'
Error while trying to remove the /opt/clump-D6KMH_ut env:
No such file or directory

Virtualenv location:                                                                                                                            Traceback (most recent call last):
  File "/home/me/.local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/home/me/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/me/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/me/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/me/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/me/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/me/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/me/.local/lib/python3.6/site-packages/pipenv/cli.py", line 1049, in sync
    pypi_mirror=pypi_mirror,
  File "/home/me/.local/lib/python3.6/site-packages/pipenv/core.py", line 2504, in do_sync
    ensure_project(three=three, python=python, validate=False, deploy=deploy, pypi_mirror=pypi_mirror)
  File "/home/me/.local/lib/python3.6/site-packages/pipenv/core.py", line 614, in ensure_project
    path_to_python = which('python') or which('py')
  File "/home/me/.local/lib/python3.6/site-packages/pipenv/core.py", line 115, in which
    p = os.path.join(location, 'bin', command)
  File "/usr/lib/python3.6/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

[uranusjr]

IMO the initial exception is good enough (the PermissionError: [Errno 13] Permission denied one). We just need to stop right there, instead of carrying on and raise another exception later in the process.

[OrangeDog]

"Error while trying to remove [...] No such file or directory" is also not great, it makes it look like the virtualenv needs to already exist for sync to work, whereas in this case it's expected that it doesn't exist.

[uranusjr]

Indeed, that’s why I said we should stop right after the permission denied line. (Or may even earlier.)

One way to do that would be to create the directory first, and then tell virtualenv to initialise inside it. This would enable us to fail early without worrying whether virtualenv has installed stuff into the target directory.

[aidin36]

Seems that it fixed in the master. I received this error using latest code from master, with python3.7 on Gnu/Linux:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.7/site-packages/virtualenv.py", line 2343, in <module>
    main()
  File "/usr/local/lib/python3.7/site-packages/virtualenv.py", line 712, in main
    symlink=options.symlink)
  File "/usr/local/lib/python3.7/site-packages/virtualenv.py", line 927, in create_environment
    site_packages=site_packages, clear=clear, symlink=symlink))
  File "/usr/local/lib/python3.7/site-packages/virtualenv.py", line 1112, in install_python
    mkdir(lib_dir)
  File "/usr/local/lib/python3.7/site-packages/virtualenv.py", line 324, in mkdir
    os.makedirs(path)
  File "/usr/local/lib/python3.7/os.py", line 211, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/local/lib/python3.7/os.py", line 211, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/local/lib/python3.7/os.py", line 221, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/opt/foo-r-OLon9H-3.7'

[uranusjr]

Still could be better, we can catch the permission error.

[Marto32]

@uranusjr Is anyone working on this? If not, mind if I give it a try?

[uranusjr]

@Marto32 Go ahead!

[AdamGold]

@uranusjr Python catches the exception inside subprocess and outputs it to stderr. From what I've seen (maybe I'm wrong), our only option is to stop the process when we get a non-zero return code in subprocess.check_output or maybe just check if "PermissionError: [Errno 13]" is inside c.err (misc.py) though it seems dirty.

[uranusjr]

A conservative check for PermissionError at the last line is practical enough IMO, although it’d probably be good to submit a similar patch to virtualenv to make it emit a sensible error on its own.

[AdamGold]

It's a good idea. Do you think maybe it's better not to change it in this case? I mean, the PermissionError is being printed to the user, caught by the returncode condition (returncode isn't 0).. Maybe it's good enough.

[uranusjr]

The thing I wonder is whether we should hide the traceback. Users tend to think it is a bug in the software and report it. We don’t want that perception.

[AdamGold]

https://github.com/pypa/pipenv/pull/3463/files

Any more known errors you can think of?