Unable to find venv/lib/python2.7/config/Makefile

[dctrwatson]

Seems the path is getting mangled. It's supposed to be looking for /usr/local/myapp/venv/lib/python2.7/config/Makefile but is actually looking for /usr/myapp/venv/lib/python2.7/config/Makefile (dropping the local)

• Ubuntu 12.04
• Python 2.7.3
• pip 1.4
• virtualenv 1.7.1.2

Traceback (most recent call last):
  File "/usr/local/myapp/venv/bin/pip", line 9, in <module>
    load_entry_point('pip==1.4', 'console_scripts', 'pip')()
  File "/usr/local/myapp/venv/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 337, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/local/myapp/venv/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 2279, in load_entry_point
    return ep.load()
  File "/usr/local/myapp/venv/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 1989, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/usr/local/myapp/venv/local/lib/python2.7/site-packages/pip/__init__.py", line 11, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar  # noqa
  File "/usr/local/myapp/venv/local/lib/python2.7/site-packages/pip/vcs/subversion.py", line 4, in <module>
    from pip.index import Link
  File "/usr/local/myapp/venv/local/lib/python2.7/site-packages/pip/index.py", line 32, in <module>
    from pip.wheel import Wheel, wheel_ext, wheel_setuptools_support, setuptools_requirement
  File "/usr/local/myapp/venv/local/lib/python2.7/site-packages/pip/wheel.py", line 18, in <module>
    from pip import pep425tags
  File "/usr/local/myapp/venv/local/lib/python2.7/site-packages/pip/pep425tags.py", line 98, in <module>
    supported_tags = get_supported()
  File "/usr/local/myapp/venv/local/lib/python2.7/site-packages/pip/pep425tags.py", line 61, in get_supported
    soabi = sysconfig.get_config_var('SOABI')
  File "/usr/lib/python2.7/sysconfig.py", line 577, in get_config_var
    return get_config_vars().get(name)
  File "/usr/lib/python2.7/sysconfig.py", line 476, in get_config_vars
    _init_posix(_CONFIG_VARS)
  File "/usr/lib/python2.7/sysconfig.py", line 344, in _init_posix
    raise IOError(msg)
IOError: invalid Python installation: unable to open /usr/myapp/venv/lib/python2.7/config/Makefile (No such file or directory)

[qwcode]

cc @dholth maybe a corner case here, but a try/except around these sysconfig calls may be in order.
pip didn't cause this per se, but we're exposing it.

note that virtualenv patches sysconfig, but that patch logic has been static, even going back to v1.7 it seems.
https://github.com/pypa/virtualenv/commits/develop/virtualenv_embedded/distutils-init.py

maybe it's just a sysconfig bug, or a debian bug. not sure

[dholth]

That variable isn't even important/available until Python 3.3ish. You won't lose anything by guarding it.

There is a second use of sysconfig in that module but it's just to get the python version.

[qwcode]

fixed in #1095

[benliles]

Not fixed in 1.4.1

[qwcode]

@benliles trace? we added a catch for IOError with a test in 1.4.1.

[benliles]

Traceback (most recent call last):
File "/usr/local/mediacore/setup.py", line 124, in
**extra_arguments_for_setup
File "/usr/lib/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/local/mediacore/venv/local/lib/python2.7/site-packages/setuptools/dist.py", line 265, in init
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/usr/local/mediacore/venv/local/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/usr/local/mediacore/venv/local/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
dist = best[req.key] = env.best_match(req, self, installer)
File "/usr/local/mediacore/venv/local/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
return self.obtain(req, installer) # try and download/install
File "/usr/local/mediacore/venv/local/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
return installer(requirement)
File "/usr/local/mediacore/venv/local/lib/python2.7/site-packages/setuptools/dist.py", line 337, in fetch_build_egg
cmd.ensure_finalized()
File "/usr/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
self.finalize_options()
File "/usr/local/mediacore/venv/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 227, in finalize_options
prefix, exec_prefix = get_config_vars('prefix', 'exec_prefix')
File "/usr/lib/python2.7/sysconfig.py", line 476, in get_config_vars
_init_posix(_CONFIG_VARS)
File "/usr/lib/python2.7/sysconfig.py", line 344, in _init_posix
raise IOError(msg)
IOError: invalid Python installation: unable to open /usr/mediacore/venv/lib/python2.7/config/Makefile (No such file or directory)

[qwcode]

@benliles this is different. this is a trace from setuptools (not pip) related to prefix. it just happens to have the same resulting message. we can't catch and handle this in pip.

[dmckean]

Something in sysconfig in python 2.7 seems to be at fault. Try calling sysconfig.get_config_var('SOABI') a second time in pep425tags.py and entirely ignore the first call. The first call reports the error, the second call behaves normally.

Also reported in virtualenv's issue tracker: #118.

My fix:

    try:
        soabi = sysconfig.get_config_var('SOABI')
    except:
        pass
    try:
        soabi = sysconfig.get_config_var('SOABI')
    except IOError as e: # Issue #1074
        warnings.warn("{0}".format(e), RuntimeWarning)
        soabi = None

[nicksloan]

This issue is caused by a patch applied by Ubuntu that specifically replaced /usr/local with /usr in several functions. Line 63404 in https://launchpadlibrarian.net/151545955/python2.7_2.7.3-5ubuntu4.3.diff.gz.

This stinks of taking a shortcut to support Ubuntu's deviations from the typical posix_local schema. Annoying.

For future reference: moving your virtualenv out of /usr/local... will fix it.

[Neetuj]

running into the same issue .. any pointers ?

root@njain2:/usr/local/horizon# pip --version
pip 1.5.4 from /usr/local/lib/python2.7/dist-packages (python 2.7)
root@njain2:/usr/local/horizon# python --version
Python 2.7.3
root@njain2:/usr/local/horizon# virtualenv --version
1.7.1.2

File "/usr/lib/python2.7/sysconfig.py", line 344, in _init_posix

raise IOError(msg)

IOError: invalid Python installation: unable to open /usr/horizon/.venv/lib/python2.7/config/Makefile (No such file or directory)

---

Cleaning up...
Command python setup.py egg_info failed with error code 1 in /usr/local/horizon/.venv/build/cryptography
Storing debug log for failure in /root/.pip/pip.log