Spaces in Python path make pip-installed launchers fail on Windows

[xflr6]

Launchers (console_scripts) installed via pip under Windows fail when Python is installed in a path that contains spaces (e.g. C:\Program Files (x86)\..., which will be the default directory for 3.5):

$ pip install vanity
$ vanity
Failed to create process.

It does work, when the package is installed manually via python setup.py install. Also, both ways install the same <packagename>.exe launcher (so this is different from #1997 and #1999).

Via pip install, the shebang in <packagename>-script.py is
#!C:\Program Files (x86)\Python27\python.exe
instead of (via python setup.py install)
#!"C:\Program Files (x86)\Python27\python.exe"

Note that both launcher scripts actually work when executed directly: only the launcher .exe seems to be more strict about quoting. So it may be good to (also) make the launcher more forgiving (is the bug tracker for distlib non-public?).

Is this similar to http://stackoverflow.com/questions/24627525/fatal-error-in-launcher-unable-to-create-process-using-c-program-files-x86?

[xflr6]

The error message is different. See issue number 1997 and 1999 (linked above) for the bug related to that SO-thread (which IIUC was was fixed by new .exe-launchers in distlib here).

In this case, the pip seems to write a non-executable shebang line (in contrast to python setup.py install).

[xflr6]

Note that this only happens with sdist installs (not with wheels).

As 7.0 switched to building wheels for sdists, one now needs pip install --no-binary <package> <package> to reproduce.

[johnthagen]

Edit: Want to point out this was run under Windows 10.
I am having the same problem, only it manifests itself because the user name has a space in it.

C:\Users>python -V
Python 3.5.1

C:\Users>pip list
pip (7.1.2)
setuptools (18.2)

C:\Users>pip -V
pip 7.1.2 from c:\users\john hagen\appdata\local\programs\python\python35\lib\site-packages (python 3.5)

C:\Users>pip install cpplint
Collecting cpplint
  Using cached cpplint-0.0.6.tar.gz
Installing collected packages: cpplint
  Running setup.py install for cpplint
Successfully installed cpplint-0.0.6

C:\Users>where.exe cpplint
C:\Users\John Hagen\AppData\Local\Programs\Python\Python35\Scripts\cpplint.exe

C:\Users>cpplint
failed to create process.

Related stackoverflow posts:

• https://stackoverflow.com/questions/31808180/installing-pyinstaller-via-pip-leads-to-failed-to-create-process?lq=1
• https://stackoverflow.com/questions/17373265/pip-easy-install-failure-failed-to-create-process

[rlucas69]

The issue here appears to be the same as one I've experienced with pip 1.5.4 on OS X 10.9.5. It is not a Windows bug per se.

The "shebang" line (hash-bang, or #!) specifying the interpreter to use must not contain a space. The workaround might be to create a symlink which doesn't have a space in the path.

Sadly, the issue is not trivially remediable by mere quoting. See the attached tarball if you want a demo (read the README) or see https://lists.gnu.org/archive/html/bug-bash/2008-05/msg00053.html

shebang_space_bug_demo.tar.gz

---

Edit: as Janzert indicates below, the problem I reproduce appears not to be related to the Windows problem of the instant issue. This comment pertains to UNIX-like OS. Same symptom with different causes -- leaving this comment above intact to help future readers disambiguate where to be spending time.

[Janzert]

Remember Windows itself doesn't do anything with shebang lines. So it is handled by the launcher executable from distlib directly.

A quick skim of the launcher source seems to show that it does explicitly and correctly handle double quotes around the interpreter command.

[tomduck]

I have a workaround.

I encountered the same problem in my pandoc-eqnos project. See Issue #6. It occurs on Windows 10 installations, but not 7. I haven't tested 8.

As others have said: A Failed to create process error arises for any console_script when the python path has a space in it. For unknown reasons, pip installs console scripts with unquoted spaced shebang lines on Windows 10. Using python setup.py install on its own has no such problem.

I implemented the workaround in my setup.py. It uses hooks provided by distutils/setuptools to quote shebang lines as needed. Note that there was an additional bug (presumably also arising from pip) that needed to be addressed.

[tomduck]

Here is the code for the workaround. You can copy this into your setup.py and enable it by hooking cmdclass into setup().

import ez_setup
ez_setup.use_setuptools()

from setuptools import setup, dist
from setuptools.command.install import install
from setuptools.command.install_scripts import install_scripts

#-----------------------------------------------------------------------------
# Hack to overcome pip/setuptools problem on Win 10.  See:
#   https://github.com/tomduck/pandoc-eqnos/issues/6
#   https://github.com/pypa/pip/issues/2783
# Note that cmdclass must be be hooked into setup().

# Custom install command class for setup()
class custom_install(install):
    """Ensures setuptools uses custom install_scripts."""
    def run(self):
        super().run()

# Custom install_scripts command class for setup()
class install_scripts_quoted_shebang(install_scripts):
    """Ensure there are quotes around shebang paths with spaces."""
    def write_script(self, script_name, contents, mode="t", *ignored):
        shebang = str(contents.splitlines()[0])
        if shebang.startswith('#!') and ' ' in shebang[2:].strip() \
          and '"' not in shebang:
            quoted_shebang = '#!"%s"' % shebang[2:].strip()
            contents = contents.replace(shebang, quoted_shebang)
        super().write_script(script_name, contents, mode, *ignored)

# The custom command classes only need to be used on Windows machines
if os.name == 'nt':
    cmdclass = {'install': custom_install,
                'install_scripts': install_scripts_quoted_shebang},

    # Below is another hack to overcome a separate bug.  The
    # dist.Distribution.cmdclass dict should not be stored in a length-1 list.

    # Save the original method
    dist.Distribution._get_command_class = dist.Distribution.get_command_class

    # Define a new method that repairs self.cmdclass if needed
    def get_command_class(self, command):
        """Pluggable version of get_command_class()"""
        try:
            # See if the original behaviour works
            return dist.Distribution._get_command_class(self, command)
        except TypeError:
            # If self.cmdclass is the problem, fix it up
            if type(self.cmdclass) is tuple and type(self.cmdclass[0]) is dict:
                self.cmdclass = self.cmdclass[0]
                return dist.Distribution._get_command_class(self, command)
            else:
                # Something else went wrong
                raise

    # Hook in the new method
    dist.Distribution.get_command_class = get_command_class

else:
    cmdclass = {}