Pip locks up after failing to download a package due to network error.

[Euphorbium]

• Pip version: 9.0.1
• Python version: 3.5.2
• Operating system: Mac

Description:

If you try to install a package from git repository, with package pinned to a tag or a commit, and this fails due to network issues, all other attempts give

Obtaining django from git+https://github.com/Euphorbium/django.git@reverse-multithreading#egg=django (from -r requirements.txt (line 6))
  Complete output from command git rev-parse HEAD:
  fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
  Use '--' to separate paths from revisions, like this:
  'git <command> [<revision>...] -- [<file>...]'
  HEAD
  
  ----------------------------------------
Command "git rev-parse HEAD" failed with error code 128 in /pipeline/source/src/django

removing the cache directory does not help.

What I've run:

pip install --cache-dir ${pip_download_cache} -r requirements.txt

[pradyunsg]

Hey! Thanks for filing the bug report! :)

Could you post the entire log when running pip install --verbose --cache-dir ${pip_download_cache} -r requirements.txt?

[Euphorbium]

The command was different. It was pip install --verbose --exists-action w --cache-dir ${pip_download_cache} -r requirements.txt Here is verbose output.

Command "git rev-parse HEAD" failed with error code 128 in /pipeline/source/src/django
Exception information:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/local/lib/python3.6/site-packages/pip/commands/install.py", line 335, in run
    wb.build(autobuilding=True)
  File "/usr/local/lib/python3.6/site-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_set.py", line 516, in _prepare_file
    req_to_install.update_editable(not self.is_download)
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_install.py", line 586, in update_editable
    vcs_backend.obtain(self.source_dir)
  File "/usr/local/lib/python3.6/site-packages/pip/vcs/git.py", line 138, in obtain
    if self.check_destination(dest, url, rev_options, rev_display):
  File "/usr/local/lib/python3.6/site-packages/pip/vcs/__init__.py", line 214, in check_destination
    if not self.check_version(dest, rev_options):
  File "/usr/local/lib/python3.6/site-packages/pip/vcs/git.py", line 106, in check_version
    return self.get_revision(dest).startswith(rev_options[0])
  File "/usr/local/lib/python3.6/site-packages/pip/vcs/git.py", line 171, in get_revision
    ['rev-parse', 'HEAD'], show_stdout=False, cwd=location)
  File "/usr/local/lib/python3.6/site-packages/pip/vcs/__init__.py", line 325, in run_command
    spinner)
  File "/usr/local/lib/python3.6/site-packages/pip/utils/__init__.py", line 707, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command "git rev-parse HEAD" failed with error code 128 in /pipeline/source/src/django
Looking up "https://pypi.python.org/pypi/pip/json" in the cache
Current age based on date: 108446
Freshness lifetime from max-age: 600
The cached response is "stale" with no etag, purging
Starting new HTTPS connection (1): pypi.python.org
"GET /pypi/pip/json HTTP/1.1" 200 72983
Updating cache with response from "https://pypi.python.org/pypi/pip/json"
Caching b/c date exists and max-age > 0

[Euphorbium]

Oh, and operating system is probably not mac, this is running in docker container based on ubuntu.

[pradyunsg]

What's your requirements.txt like?

[Euphorbium]

I have just tried to run it with a simplified requirements file, containing only
-e git+https://github.com/onitsoft/django.git@reverse-multithreading#egg=django
here is the output.

Obtaining django from git+https://github.com/onitsoft/django.git@reverse-multithreading#egg=django (from -r requirements.txt (line 1))
  Running command git config --get-regexp remote\..*\.url
  remote.origin.url https://github.com/onitsoft/django.git
  Clone in ./src/django exists, and has correct URL (https://github.com/onitsoft/django.git)
  Running command git rev-parse HEAD
  fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
  Use '--' to separate paths from revisions, like this:
  'git <command> [<revision>...] -- [<file>...]'
  HEAD
Cleaning up...
Command "git rev-parse HEAD" failed with error code 128 in /pipeline/source/src/django
Exception information:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/local/lib/python3.6/site-packages/pip/commands/install.py", line 335, in run
    wb.build(autobuilding=True)
  File "/usr/local/lib/python3.6/site-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_set.py", line 516, in _prepare_file
    req_to_install.update_editable(not self.is_download)
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_install.py", line 586, in update_editable
    vcs_backend.obtain(self.source_dir)
  File "/usr/local/lib/python3.6/site-packages/pip/vcs/git.py", line 138, in obtain
    if self.check_destination(dest, url, rev_options, rev_display):
  File "/usr/local/lib/python3.6/site-packages/pip/vcs/__init__.py", line 214, in check_destination
    if not self.check_version(dest, rev_options):
  File "/usr/local/lib/python3.6/site-packages/pip/vcs/git.py", line 106, in check_version
    return self.get_revision(dest).startswith(rev_options[0])
  File "/usr/local/lib/python3.6/site-packages/pip/vcs/git.py", line 171, in get_revision
    ['rev-parse', 'HEAD'], show_stdout=False, cwd=location)
  File "/usr/local/lib/python3.6/site-packages/pip/vcs/__init__.py", line 325, in run_command
    spinner)
  File "/usr/local/lib/python3.6/site-packages/pip/utils/__init__.py", line 707, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command "git rev-parse HEAD" failed with error code 128 in /pipeline/source/src/django
Looking up "https://pypi.python.org/pypi/pip/json" in the cache
Current age based on date: 57057
Freshness lifetime from max-age: 600
The cached response is "stale" with no etag, purging
Starting new HTTPS connection (1): pypi.python.org
"GET /pypi/pip/json HTTP/1.1" 200 72983
Updating cache with response from "https://pypi.python.org/pypi/pip/json"
Caching b/c date exists and max-age > 0

This easily reproducible. Try to install it, then make a mess of your internet connection. After that even when having a stable internet connection, it is never possible to install the package.

[thebigmunch]

I've run into the same issue. Not sure if the actual packages matter or, if as suggested, it's a symptom of some network or other issue. But the packages involved for me so far are: bidict, filetype.py, pycodestyle. I've used pip 9.0.1, 9.0.2, and 9.0.3.

However, deleting pip's cache directory DID result in proper installation for me. I.e. not a custom cache directory

[thebigmunch]

From poking, on my system, it seems to be caused by the lockfile infinite looping problems posted in some other issues. I get a traceback ending in the time.sleep call in lockfile.

[chrahunt]

Issues related to the self-check state locking will be satisfied by #6954.