Skip to content

Conversation

@frostming
Copy link
Contributor

@frostming frostming commented Oct 27, 2020

Thank you for contributing to Pipenv!

The issue

Close #2681
Close #1914
Close #3827
Close #2873
Close #4430

The fix

Previously, package hash values are calculated based on the file content, so all matched wheels and sdists need to be actually downloaded. For packages like tensorflow and numpy, this is a great overhead to lock dependencies, whose average file size is ~200MB.

With this patch, Pipenv will retrieve hash value directly from the URL fragment when possible.

The checklist

  • Associated issue
  • A news fragment in the news/ directory to describe this fix with the extension .bugfix, .feature, .behavior, .doc. .vendor. or .trivial (this will appear in the release changelog). Use semantic line breaks and name the file after the issue number or the PR #.

@frostming frostming added Category: Dependency Resolution Issue relates to dependency resolution. Category: Performance Issue relates to performance labels Oct 27, 2020
@uranusjr
Copy link
Member

Do we want to submit this to upstream pip-tools as well?

@frostming
Copy link
Contributor Author

let me check, since the whole logic is added by pipenv

@frostming
Copy link
Contributor Author

@uranusjr Confirmed, the latest version of piptools is already reading hashes from PyPI JSON API. So I would like to keep this change in Pipenv only

@uranusjr
Copy link
Member

Cool. I’m curious why they chose to use the JSON API, but 🤷

@frostming frostming merged commit 912164d into master Oct 28, 2020
@frostming frostming deleted the url-hash branch October 28, 2020 06:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Category: Dependency Resolution Issue relates to dependency resolution. Category: Performance Issue relates to performance

Projects

None yet

3 participants