Pipenv updates pinned packages in Pipfile.lock when there is an editable git dependency with unpinned sub-deps

[weiji14]

Hi there,

First off, great work on dependabot, it's really awesome being able to keep my packages up to date easily, and have the changelogs documented in Pull Requests 😄

Anyways, I'm just reporting a strange issue with how dependabot handles the update of 'dev-packages' for Python pipenv packages. This commit weiji14/deepbedmap@59c7e58 which bumps pytest from 3.10.0 to 4.0.1 also updates some of my other non-dev 'packages' in the Pipenv.lock file. It's currently trying to update numpy from 1.14.5 to 1.15.4 and netcdf4 from 1.4.1 to 1.4.2 even though those two packages are pinned in the Pipfile!

Normally I'm all good with having the latest and greatest, but this seems to be unexpected behaviour that might affect others. Expected behaviour should be that the pull request respects the pinned non-dev packages in the Pipfile.

I.e. a Pipfile.lock like this instead of this. This diff between the two should show how the numpy and netcdf4 package versions are preserved.

Happy to provide further information if needed 😁

Update: Upstream issue with pipenv, see comments below.

[greysteil]

Thanks for reporting - that's definitely a bug! I'll take a look now.

[weiji14]

Cool, I'm gonna manually update those packages for now :)

[greysteil]

👍 , or if you give me 10 minutes I can probably get it fixed and rebase them from my side.

[weiji14]

No rush, it's close to midnight on my end and I just wanted to kick off some CI builds before bed 😃

[greysteil]

Perhaps for the best. I'd assumed this was a bug in the way Dependabot was preparing Pipfiles for updates, but it looks more likely it's a Pipenv issue. I'll get to the bottom of it and get it fixed either way.

[weiji14]

I see, not surprised actually. I'm using pipenv version 2018.10.13, though I did notice 2018.11.14 came out a week ago. I remember the 2018.10.9 pipenv update giving me quite a headache so I'm purposely holding back from upgrading pipenv until all the bugs get ironed out. ☺️

[greysteil]

Yeah, this is a Pipenv bug. I'll give the team a heads up and put together a replication case. I think it's struggling with resolution for dependencies that are sub-deps of the git dependency you have in your Pipfile (as well as top-level dependencies).

[greysteil]

FYI, I've replicated and created this issue. Hopefully we can get it fixed quickly. Thanks for making Dependabot and Pipenv better!

[weiji14]

Yeah, this is a Pipenv bug. I'll give the team a heads up and put together a replication case. I think it's struggling with resolution for dependencies that are sub-deps of the git dependency you have in your Pipfile (as well as top-level dependencies).

Ah yes, I think I saw a pipenv issue related to that a little while back, but can't seem to find it anymore. Just noticed it's not restricted to dev-packages, this new Cython bump commit weiji14/deepbedmap@4cace4f is also bumping up numpy and netcdf4. So you're right that it is to do with the git dependency not having pinned sub-dependencies

FYI, I've replicated and created this issue. Hopefully we can get it fixed quickly. Thanks for making Dependabot and Pipenv better!

I'll close this since it's an upstream issue. Will take extra care with my python packages in the meantime. Thanks for the the quick feedback!

[weiji14]

Just noting that pypa/pipenv#3296 has been closed with pypa/pipenv#3297 and pipenv v2018.11.26 has been released.

[greysteil]

Yes! Should be working just fine now. pypa/pipenv#3304 will bring even more improvements here as it will stop Dependabot from updating unrelated sub-dependencies. 🎉

[techalchemy]

as a related FYI that branch will be kept up with master because I use it all the time...