--skip-lock: tomlkit.exceptions.NonExistentKey: 'Key "source" does not exist.'

[schuhschuh]

Issue description

When using pipenv install --skip-lock, I get the error

tomlkit.exceptions.NonExistentKey: 'Key "source" does not exist.'

because there is no [[source]] defined in the Pipfile.

The culprit is the following line of code from pipenv.project.Project.get_lockfile_meta:

sources = [dict(source) for source in self.parsed_pipfile["source"]]

which requires source key to be present in parsed Pipfile.

Workaround, add default [[source]] section for PyPI to your Pipfile:

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

Expected result

A [[source]] section should not be required and installation proceed as normal.

Actual result

Installing dependencies from Pipfile…
Traceback (most recent call last):
  File "/opt/miniconda3/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/opt/miniconda3/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/opt/miniconda3/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/opt/miniconda3/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/miniconda3/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/miniconda3/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/opt/miniconda3/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/opt/miniconda3/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/opt/miniconda3/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/opt/miniconda3/lib/python3.7/site-packages/pipenv/cli/command.py", line 254, in install
    editable_packages=state.installstate.editables,
  File "/opt/miniconda3/lib/python3.7/site-packages/pipenv/core.py", line 1874, in do_install
    keep_outdated=keep_outdated
  File "/opt/miniconda3/lib/python3.7/site-packages/pipenv/core.py", line 1253, in do_init
    pypi_mirror=pypi_mirror,
  File "/opt/miniconda3/lib/python3.7/site-packages/pipenv/core.py", line 795, in do_install_dependencies
    lockfile = project.get_or_create_lockfile(from_pipfile=True)
  File "/opt/miniconda3/lib/python3.7/site-packages/pipenv/project.py", line 754, in get_or_create_lockfile
    lockfile_dict.update({"_meta": self.get_lockfile_meta()})
  File "/opt/miniconda3/lib/python3.7/site-packages/pipenv/project.py", line 790, in get_lockfile_meta
    sources = [dict(source) for source in self.parsed_pipfile["source"]]
  File "/opt/miniconda3/lib/python3.7/site-packages/pipenv/vendor/tomlkit/container.py", line 500, in __getitem__
    raise NonExistentKey(key)
tomlkit.exceptions.NonExistentKey: 'Key "source" does not exist.'

Steps to replicate

Example Pipfile:

[packages]
pytest = "*"
tox = "*"

Run pipenv install --skip-lock.

---

Pipenv version: '2018.11.26'

[frostming]

Fixed in master, close it now. Let's wait for the coming release

[ddudnik]

Hey, looks like this issue is back in 2020.6.2
Caught the same error when running pipenv install --skip-lock and when requirements.txt file was present so that it's used instead of a Pipfile

[dem1tris]

Same. 2020.6.2 shows this when trying to use requirements.txt.

$ pipenv install -r requirements.txt --skip-lock

[Kristinita]

Status: Still doesn’t work for me for Pipenv 2020.8.13 ❌

1. MCVE

See the KiraSkipLock branch of my demo/debugging repository.

• Pipfile:

  [dev-packages]
  pylint = "*"

• A part of the .travis.yml:

  install:
  - export PIPENV_SKIP_LOCK=1
  - pip install pipenv
  - pipenv --version
  
  script:
  - pipenv install --dev
  

2. Behavior

2.1. Current

• Travis build:

  Traceback (most recent call last):
  
    File "/home/travis/virtualenv/python3.9.0/bin/pipenv", line 8, in <module>
  
      sys.exit(cli())
  
    File "/home/travis/virtualenv/python3.9.0/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 829, in __call__
  
      return self.main(*args, **kwargs)
  
    File "/home/travis/virtualenv/python3.9.0/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 782, in main
  
      rv = self.invoke(ctx)
  
    File "/home/travis/virtualenv/python3.9.0/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 1259, in invoke
  
      return _process_result(sub_ctx.command.invoke(sub_ctx))
  
    File "/home/travis/virtualenv/python3.9.0/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
  
      return ctx.invoke(self.callback, **ctx.params)
  
    File "/home/travis/virtualenv/python3.9.0/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
  
      return callback(*args, **kwargs)
  
    File "/home/travis/virtualenv/python3.9.0/lib/python3.9/site-packages/pipenv/vendor/click/decorators.py", line 73, in new_func
  
      return ctx.invoke(f, obj, *args, **kwargs)
  
    File "/home/travis/virtualenv/python3.9.0/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
  
      return callback(*args, **kwargs)
  
    File "/home/travis/virtualenv/python3.9.0/lib/python3.9/site-packages/pipenv/vendor/click/decorators.py", line 21, in new_func
  
      return f(get_current_context(), *args, **kwargs)
  
    File "/home/travis/virtualenv/python3.9.0/lib/python3.9/site-packages/pipenv/cli/command.py", line 232, in install
  
      retcode = do_install(
  
    File "/home/travis/virtualenv/python3.9.0/lib/python3.9/site-packages/pipenv/core.py", line 2052, in do_install
  
      do_init(
  
    File "/home/travis/virtualenv/python3.9.0/lib/python3.9/site-packages/pipenv/core.py", line 1305, in do_init
  
      do_install_dependencies(
  
    File "/home/travis/virtualenv/python3.9.0/lib/python3.9/site-packages/pipenv/core.py", line 827, in do_install_dependencies
  
      lockfile = project.get_or_create_lockfile(from_pipfile=True)
  
    File "/home/travis/virtualenv/python3.9.0/lib/python3.9/site-packages/pipenv/project.py", line 783, in get_or_create_lockfile
  
      lockfile_dict.update({"_meta": self.get_lockfile_meta()})
  
    File "/home/travis/virtualenv/python3.9.0/lib/python3.9/site-packages/pipenv/project.py", line 819, in get_lockfile_meta
  
      sources = [dict(source) for source in self.parsed_pipfile["source"]]
  
    File "/home/travis/virtualenv/python3.9.0/lib/python3.9/site-packages/pipenv/vendor/tomlkit/container.py", line 538, in __getitem__
  
      raise NonExistentKey(key)
  
  tomlkit.exceptions.NonExistentKey: 'Key "source" does not exist.'
  
  The command "pipenv install --dev" exited with 1.

2.2. Expected

If:

- export PIPENV_SKIP_LOCK=1

No traceback, see the Travis build:

Pipfile.lock not found, creating…

Locking [dev-packages] dependencies…

Building requirements…

Resolving dependencies…

Success!

Locking [packages] dependencies…

Updated Pipfile.lock (4e85bb)!

Installing dependencies from Pipfile.lock (4e85bb)…

  ================================ 7/7 — 00:00:00

The command "pipenv install --dev" exited with 0.

3. Environment

1. Operating system:

   1. Windows 10.0.19041.508 Pro N for Workstations 64-bit EN (Local)
   2. Ubuntu 20.04.1 LTS (Travis CI)

2. Python 3.9.0

3. pipenv 2020.8.13

Thanks.

[Mause]

Please try with the latest version of pipenv

[Kristinita]

@Mause, I reproduce the problem for Pipenv 2020.11.15. See the Travis build. Excuse me, I didn't notice that a previous Pipenv version (2020.8.13) pre-installed on the Travis.

Thanks.

[Mause]

Your Pipfile is missing the source section - this isn't optional

[Kristinita]

@Mause , see the original @schuhschuh issue:

Expected result

A [[source]] section should not be required and installation proceed as normal.

Thanks.

[uranusjr]

It seems like the fix was lost somehow (probably due to a merge). Reopening.

https://github.com/pypa/pipenv/blob/v2020.11.15/pipenv/project.py#L820-L833

[uranusjr]

The fix to this should not be too difficult (set sources to [DEFAULT_SOURCE] if the section does not exist). A PR would be very welcomed!

[clabe45]

I don't see the issue anymore, but I can make a PR If it still needs to be fixed