pre-commit hook for pip-sync

[Berk-Tosun]

What's the problem this feature will solve?

With the pre-commit hook for pip-compile (#974), I am confident that any changes that I do in my development environment will be pinned. I would like to have the same for pip-sync so that nobody's development environment gets outdated.

In a project, it might be that one of the collaborators have modified the dependencies and pushed their new pip-compiled requirements.txt. As I get their changes via a git pull, I want to be sure that we (everyone) keep sharing the same dependencies. As far as I see with the current pip-tools workflow, pip-sync has to be run manually. That means, I have to be informed by the collaborator or check the pulled/merged commits to see if I need to run pip-sync. This could lead to some issues, could we automate pip-sync so that we avoid those?

If it gets implemented, the new hook combined with the existing pip-compile hook would allow a mostly automated dependency management.

Describe the solution you'd like

I am no expert at pre-commit, but I think it can be used for developing a new hook similar to that of pip-compile(#976). I believe one can use the post-merge stage for this hook.

.pre-commit-hooks.yaml for the new hook would something like the following:

- id: pip-sync
  name: pip-sync
  description: Automatically sync requirements.
  entry: pip-sync
  language: python
  files: ^requirements\.(in|txt)$
  args: [--ask]
  

Alternative Solutions

Simply printing a message that the environment needs to modified could be a better default option as it is not intrusive (#1076). The default hook configuration can use --dry-run for the args. Users who want automation can change in their .pre-commit-config.yaml by modifying the args from --dry-run to --ask instead.

For the dependencies which are using semantic versioning, it would ideally automatically sync changes in the minor and patch version and ask only for major version changes.

I am new to pip-tools and perhaps I am missing a point. If so, please share your existing solutions/workflows.

Additional context

https://git-scm.com/docs/githooks#_post_merge

Thank you for pip-tools.

[chrysle]

Thank you for the proposal!

Would you like to do some performance tests with your hook and open a pull request?

[Berk-Tosun]

I am going slow at the moment, but I will keep you updated.

Meanwhile could you tell me what is your expectation from the performance test in a little more detail? I checked #976 but could not see any performance tests there.

[chrysle]

Just testing that the introduced hook is fast enough (running pre-commit locally), by including it in the .pre-commit-config.yaml file. I think that should work.

[RonnyPfannschmidt]

to be fir, the ask here is not an actual pre-commit hook, but a post update hook (so every time someone updates the git checkouts of the pins, the updates propagate

[chrysle]

@RonnyPfannschmidt Thank you, I missed that! Then I think the proposed config should be modified like this:

- id: pip-sync
  name: pip-sync
  description: Automatically sync requirements.
  entry: pip-sync
  language: python
  files: ^requirements\.(in|txt)$
  args: [--ask]
  stages: [post-merge]

[hofrob]

We're using the following stages:

      - id: pip-sync
        name: pip-sync dev
        entry: ./venv/bin/pip-sync requirements_dev.txt
        language: system
        pass_filenames: false
        always_run: true
        verbose: true
        stages: [ post-checkout, post-rewrite, post-merge, post-commit ]

But to be honest, something changed which makes this extremely slow. Previously, running this command took maybe a few tenths of a second which was ok to do after every branch switch or pull. Now, running pip-sync requirements_dev.txt takes 20s+. It seems like it's doing a clean install every time.

Does anyone experience something similar?

[AndydeCleyre]

@hofrob The 7.0.0 release has a bug that causes pip-sync to do extra work, which is fixed in a release coming soon.