Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve on pip-compile to only run pip-compile a second time when the requirements.txt has been changed #10771

Open
jerbob92 opened this issue Oct 10, 2024 · 0 comments

Comments

@jerbob92
Copy link
Contributor

Code improvement description

I have done improvements on pip-compile before: #5905
pip-compile is amazingly slow since that the backtracking resolver has been made the default and I'm trying to do some speed improvements to make it better for us. Dependency updates can run for multiple hours for us (sometimes even 10+ hours). Folks at pip are also trying to make speed improvements.

One thing we could potentially improve on is only running pip-compile the second time if there's actually a change made in the requirements.txt file. If there is no change in the requirements.txt file, it means no upgrade was done, in that case we can just run write_original_manifest_files and move on, without running pip-compile again.

I also wonder if write_original_manifest_files is still necessary here, because when I run pip-compile -P {dependency} myself, it won't add it to the requirements.in file, which I believe is why that line is in there.

Relevant code is here: https://github.com/dependabot/dependabot-core/blob/main/python/lib/dependabot/python/update_checker/pip_compile_version_resolver.rb#L111

What do you think @Nishnha @jurre ?

@jerbob92 jerbob92 changed the title Improve on pip-compile to only run pip-compile a second time when the Improve on pip-compile to only run pip-compile a second time when the requirements.txt has been changed Oct 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

1 participant