Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace git subprocess by pygit2 #1045

Open
astrojuanlu opened this issue Mar 21, 2023 · 9 comments
Open

Replace git subprocess by pygit2 #1045

astrojuanlu opened this issue Mar 21, 2023 · 9 comments

Comments

@astrojuanlu
Copy link
Contributor

Is your feature request related to a problem? Please describe.
Now copier relies on git being installed in the system:

1. Install Git 2.27 or newer.

However, it would be nice if all the dependencies could be installed with pip without having to reach for the system package manager. This would be useful for folks that don't have admin access to their systems, but can pip install to their user locations.

Describe the solution you'd like
Replace subprocess calls to git through plumbum:

copier/copier/main.py

Lines 801 to 811 in 47f3d59

def _git_initialize_repo(self):
"""Initialize a git repository in the current directory."""
git("init", retcode=None)
git("add", ".")
git("config", "user.name", "Copier")
git("config", "user.email", "copier@copier")
# 1st commit could fail if any pre-commit hook reformats code
git("commit", "--allow-empty", "-am", "dumb commit 1", retcode=None)
git("commit", "--allow-empty", "-am", "dumb commit 2")
git("config", "--unset", "user.name")
git("config", "--unset", "user.email")

with pygit2, the Python bindings for libgit2, which ship precompiled binary wheels for plenty of platforms:

https://pypi.org/project/pygit2/1.11.1/#files

As a side effect, this will also probably be faster (but I don't have data to back this up).

Describe alternatives you've considered
Could not think of other alternatives.

Additional context
This was rejected for cookiecutter as early as 2016: cookiecutter/cookiecutter#846 (comment)

In Kedro we are trying to remove our dependency with system git kedro-org/kedro#2051 which prompted me to open this issue.

@sisp
Copy link
Member

sisp commented Mar 21, 2023

I like the idea of removing the prerequisite of having Git installed. But I stumbled over one caveat that is a blocker for me:

Binary wheels for Windows are available, but they don’t have support for ssh.

https://www.pygit2.org/install.html#quick-install (under "Caveats")

I don't use Windows, but I know many who do, and supporting SSH is quite important in our environment.

@sisp
Copy link
Member

sisp commented Mar 21, 2023

But perhaps dulwich could be an alternative worth looking into. I have no experience with dulwich, but, e.g., DVC uses it in their scmrepo library as one of the possible Git library backends.

dulwich uses a dual license:

License: Apache License, version 2 or GNU General Public License, version 2 or later.

I'm no laywer, but since both Apache v2+ and GPL v2+ are possible, I think it should be fine (mainly looking at Apache v2+ here).

WDYT, @astrojuanlu?

@pawamoy
Copy link
Contributor

pawamoy commented Mar 21, 2023

I'd suggest that we implement this with a fallback to the git system exec, and maybe an option to prefer the system exec over the Python lib/bindng, since they might differ in features/behavior.

@astrojuanlu
Copy link
Contributor Author

I havw no experience with either, but @pawamoy suggestion looks sensible enough!

@astrojuanlu
Copy link
Contributor Author

Just to be sure: is this issue open for pull requests then? Or is there more discussion needed?

@yajo
Copy link
Member

yajo commented Apr 1, 2023

I think this can be split in 2.

The 1st thing is to not require git. Copier can work with gitless templates. They have less features, but they work. Git should only be required when using a gitful template.

If you want a gitful template, I can't imagine why would you not have git installed, so that could be the real fix here.

If there are reasons why that can happen and still dulwitch works for us (copier uses git deeply and intensely), then replacing git is the 2nd part of the story.

FWIW if we're about letting users use gitful templates without git, another option is to support templates in zip files.

Also we can bundle git propagated in the nix package, so installing copier with nix just works out of the box.

@sisp
Copy link
Member

sisp commented Apr 1, 2023

The 1st thing is to not require git. Copier can work with gitless templates. They have less features, but they work. Git should only be required when using a gitful template.

Related to #312.

@astrojuanlu
Copy link
Contributor Author

For reference, part (1) of @yajo's comment above (gh-312) was implemented in gh-1074 👍🏽

@yajo
Copy link
Member

yajo commented Aug 29, 2023

Maybe still there's work to do. See #1304.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants