JSONDecodeError while locking if pythonwarning is tirggered

[Maks3w]

Issue description

With PYTHONWARNINGS=once pipenv fails because warning is appended to JSON generated lock

Actual result

[{"version": "18.2.0", "hashes": ["sha256:10cbf6e27dbce8c30807caf056c8eb50917e0eaafe86347671b57254......, {"version": "4.0.0", "index": "pypi", "hashes": ["sha256:488c842647bbeb350029da10325cb40af0a9c7a2fdda45aeb1dda75b60048ffb", "sha256:c055690dfefa744992f563e8c3a654089a6aa5b8092dded9b6fafbd70b2e45a7"], "name": "pytest"}]/usr/local/lib/python3.6/site-packages/pipenv/patched/notpip/_vendor/msgpack/fallback.py:222: PendingDeprecationWarning: encoding is deprecated, Use raw=False instead.
  PendingDeprecationWarning)


Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 636, in venv_resolve_deps
    results = json.loads(results.split("RESULTS:")[1].strip())
  File "/usr/local/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.6/json/decoder.py", line 342, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 9648 (char 9647)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/cli/command.py", line 325, in lock
    pypi_mirror=state.pypi_mirror,
  File "/usr/local/lib/python3.6/site-packages/pipenv/core.py", line 1047, in do_lock
    lockfile=lockfile
  File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 649, in venv_resolve_deps
    raise RuntimeError("There was a problem with locking.")
RuntimeError: There was a problem with locking.

Workaround is set PYTHONWARNINGS=ignore

[duplicate-issues]

Hey @Maks3w,

We did a quick check and this issue looks very darn similar to

• #3271 - PEEP: Allow per-project pipenv configuration
• #3270 - emit warning when using VCS dependency without editable flag
• #3269 - Latest pythonfinder logic cant handle pyenv and --two combination

This could be a coincidence, but if any of these issues solves your problem then I did a good job 😄

If not, the maintainers will get to this issue shortly.

Cheers,
Your Friendly Neighborhood ProBot

[techalchemy]

Hm the solution is to get warnings to write to stderr...

[rdpoor]

Probably a cockpit error, but I'm still seeing this bug. Abbreviated summary form here, full details in the attached files:

pipenv_bug_3273.txt
pipenv_support_3273.txt

PS C:\Users\myuser\rdp\SmartCableSensors> py -m pip install pipenv --upgrade
Requirement already up-to-date: pipenv in c:\users\myuser\appdata\local\programs\python\python37-32\lib\site-packages (2018.11.26)
...
PS C:\Users\myuser\rdp\SmartCableSensors> py -m pipenv update
...
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\myuser\\AppData\\Local\\Temp\\tmp3s1yby1psource\\mcp2210\
\.git\\objects\\01\\09b9f646885fd3e90a734f158e1c416a06fb11'
...
json.decoder.JSONDecodeError: Extra data: line 1 column 8302 (char 8301)
During handling of the above exception, another exception occurred:
RuntimeError: There was a problem with locking.

[techalchemy]

The fix hasn’t been released so you’ll have to install from master. Can you confirm this is using the master branch?

[rdpoor]

The fix hasn’t been released so you’ll have to install from master.

That's almost certainly the problem. I assumed that since the version on my machine had a version later than when you closed this bug that I had the latest:

PS E:\github\SmartCableSensors> pipenv --version
pipenv, version 2018.11.26

I'll install from master and try again. Note to my future self (and others that follow):

 > pip install git+https://github.com/pypa/pipenv.git
Successfully installed pipenv-2018.11.27.dev0

Thank you.

P.S.: Fun fact: I dropped a suggestion to the author of commitstrip and garnered a mention of pipenv there:

http://www.commitstrip.com/en/2018/04/25/why-does-it-take-such-a-long-time

[shaqirfan]

I did install from master but now when I run pipenv install --dev it throws other bunch of errors

[shaqirfan]

C:\Users\shaqi\Documents\AdvancePython\2019fa-csci-test-shaqirfan>pipenv install --dev
Traceback (most recent call last):
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\pythonfinder\models\python.py", line 614, in parse_executable
result_version = get_python_version(path)
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\pythonfinder\utils.py", line 114, in get_python_version
raise InvalidPythonVersion("%s is not a valid python path" % path)
pipenv.vendor.pythonfinder.exceptions.InvalidPythonVersion: C:/ProgramData/Anaconda2/Scripts/python.exe is not a valid python path

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\programdata\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "c:\programdata\anaconda3\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\ProgramData\Anaconda3\Scripts\pipenv.exe_main.py", line 7, in
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\click\core.py", line 764, in call
return self.main(*args, **kwargs)
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\click\core.py", line 717, in main
rv = self.invoke(ctx)
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\click\core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\click\core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\click\core.py", line 555, in invoke
return callback(*args, **kwargs)
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\click\decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\click\core.py", line 555, in invoke
return callback(*args, **kwargs)
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\click\decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\cli\command.py", line 251, in install
site_packages=state.site_packages
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\core.py", line 1885, in do_install
site_packages=site_packages,
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\core.py", line 571, in ensure_project
pypi_mirror=pypi_mirror,
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\core.py", line 481, in ensure_virtualenv
python = ensure_python(three=three, python=python)
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\core.py", line 381, in ensure_python
path_to_python = find_a_system_python(python)
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\core.py", line 348, in find_a_system_python
python_entry = find_python(finder, line)
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\utils.py", line 2183, in find_python
result = finder.find_python_version(line)
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\pythonfinder\pythonfinder.py", line 267, in find_python_version
if os.name == "nt" and self.windows_finder is not None:
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\pythonfinder\pythonfinder.py", line 128, in windows_finder
self._windows_finder = WindowsFinder()
File "<attrs generated init 42343eac96b22fb46bfe40009b80ae880c01e0dd>", line 13, in init
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\pythonfinder\models\windows.py", line 110, in get_versions
versions[py_version.version_tuple[:5]] = base_dir
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\pythonfinder\models\python.py", line 361, in getattribute
result = super(PythonVersion, self).getattribute(key)
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\pythonfinder\models\python.py", line 426, in version_tuple
self.patch,
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\pythonfinder\models\python.py", line 371, in getattribute
instance_dict = self.parse_executable(executable)
File "C:\Users\shaqi\AppData\Roaming\Python\Python37\site-packages\pipenv\vendor\pythonfinder\models\python.py", line 616, in parse_executable
raise ValueError("Not a valid python path: %r" % path)
ValueError: Not a valid python path: 'C:/ProgramData/Anaconda2/Scripts/python.exe'

[shaqirfan]

I don't have Anaconda2 at all. All I have is Anaconda3