-
-
Couldn't load subscription status.
- Fork 960
Fix Git.execute shell use and reporting bugs #1687
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
That test is not testing whether or not a shell is used (nor does it need to test that), but just whether the library actually runs git, passes an argument to it, and returns text from its stdout.
In the Popen calls in Git.execute, for all combinations of allowed values for Git.USE_SHELL and the shell= keyword argument.
The log message shows "Popen(...)", not "execute(...)". So it should faithfully report what is about to be passed to Popen in cases where a user reading the log would otherwise be misled into thinking this is what has happened. Reporting the actual "shell=" argument passed to Popen is also more useful to log than the argument passed to Git.execute (at least if only one of them is to be logged).
Both new shell-related tests were causing the code under test to split "git" into ["g", "i", "t"] and thus causing the code under test to attempt to execute a "g" command. This passes the command as a one-element list of strings rather than as a string, which avoids this on all operating systems regardless of whether the code under test has a bug being tested for. This would not occur on Windows, which does not iterate commands of type str character-by-character even when the command is run without a shell. But it did happen on other systems. Most of the time, the benefit of using a command that actually runs "git" rather than "g" is the avoidance of confusion in the error message. But this is also important because it is possible for the user who runs the tests to have a "g" command, in which case it could be very inconvenient, or even unsafe, to run "g". This should be avoided even when the code under test has a bug that causes a shell to be used when it shouldn't or not used when it should, so having separate commands (list and str) per test case parameters would not be a sufficient solution: it would only guard against running "g" when a bug in the code under test were absent.
This also helps mock Popen over a smaller scope, which may be beneficial (especially if it is mocked in the subprocess module, rather than the git.cmd module, in the future).
Because mock.call.kwargs, i.e. the ability to examine m.call_args.kwargs where m is a Mock or MagicMock, was introduced in Python 3.8. Currently it is only in test/test_git.py that any use of mocks requires this, so I've put the conditional import logic to import mock (the top-level package) rather than unittest.mock only there. The mock library is added as a development (testing) dependency only when the Python version is lower than 3.8, so it is not installed when not needed. This fixes a problem in the new tests of whether a shell is used, and reported as used, in the Popen call in Git.execute. Those just-introduced tests need this, to be able to use mock_popen.call_args.kwargs on Python 3.7.
This updates the shell variable itself, only when it is None, from self.USE_SHELL. (That attribute is usually Git.USE_SHELL rather than an instance attribute. But although people probably shouldn't set it on instances, people will expect that this is permitted.) Now: - USE_SHELL is used as a fallback only. When shell=False is passed, USE_SHELL is no longer consuled. Thus shell=False always keeps a shell from being used, even in the non-default case where the USE_SHELL attribue has been set to True. - The debug message printed to the log shows the actual value that is being passed to Popen, because the updated shell variable is used both to produce that message and in the Popen call.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks so much for the fix which brings the behaviour in line with the documentation, and for adding regression protection.
This PR is also a wonderful example on what's possible with the unittest and mock libraries and generally what's possible with python testing.
🙏
[](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [GitPython](https://togithub.com/gitpython-developers/GitPython) | `==3.1.37` -> `==3.1.40` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>gitpython-developers/GitPython (GitPython)</summary> ### [`v3.1.40`](https://togithub.com/gitpython-developers/GitPython/compare/3.1.38...3.1.40) [Compare Source](https://togithub.com/gitpython-developers/GitPython/compare/3.1.38...3.1.40) ### [`v3.1.38`](https://togithub.com/gitpython-developers/GitPython/releases/tag/3.1.38) [Compare Source](https://togithub.com/gitpython-developers/GitPython/compare/3.1.37...3.1.38) #### What's Changed - Add missing assert keywords by [@​EliahKagan](https://togithub.com/EliahKagan) in [https://github.com/gitpython-developers/GitPython/pull/1678](https://togithub.com/gitpython-developers/GitPython/pull/1678) - Make clear every test's status in every CI run by [@​EliahKagan](https://togithub.com/EliahKagan) in [https://github.com/gitpython-developers/GitPython/pull/1679](https://togithub.com/gitpython-developers/GitPython/pull/1679) - Fix new link to license in readme by [@​EliahKagan](https://togithub.com/EliahKagan) in [https://github.com/gitpython-developers/GitPython/pull/1680](https://togithub.com/gitpython-developers/GitPython/pull/1680) - Drop unneeded flake8 suppressions by [@​EliahKagan](https://togithub.com/EliahKagan) in [https://github.com/gitpython-developers/GitPython/pull/1681](https://togithub.com/gitpython-developers/GitPython/pull/1681) - Update instructions and test helpers for git-daemon by [@​EliahKagan](https://togithub.com/EliahKagan) in [https://github.com/gitpython-developers/GitPython/pull/1684](https://togithub.com/gitpython-developers/GitPython/pull/1684) - Fix Git.execute shell use and reporting bugs by [@​EliahKagan](https://togithub.com/EliahKagan) in [https://github.com/gitpython-developers/GitPython/pull/1687](https://togithub.com/gitpython-developers/GitPython/pull/1687) - No longer allow CI to select a prerelease for 3.12 by [@​EliahKagan](https://togithub.com/EliahKagan) in [https://github.com/gitpython-developers/GitPython/pull/1689](https://togithub.com/gitpython-developers/GitPython/pull/1689) - Clarify Git.execute and Popen arguments by [@​EliahKagan](https://togithub.com/EliahKagan) in [https://github.com/gitpython-developers/GitPython/pull/1688](https://togithub.com/gitpython-developers/GitPython/pull/1688) - Ask git where its daemon is and use that by [@​EliahKagan](https://togithub.com/EliahKagan) in [https://github.com/gitpython-developers/GitPython/pull/1697](https://togithub.com/gitpython-developers/GitPython/pull/1697) - Fix bugs affecting exception wrapping in rmtree callback by [@​EliahKagan](https://togithub.com/EliahKagan) in [https://github.com/gitpython-developers/GitPython/pull/1700](https://togithub.com/gitpython-developers/GitPython/pull/1700) - Fix dynamically-set **all** variable by [@​DeflateAwning](https://togithub.com/DeflateAwning) in [https://github.com/gitpython-developers/GitPython/pull/1659](https://togithub.com/gitpython-developers/GitPython/pull/1659) - Fix small [#​1662](https://togithub.com/gitpython-developers/GitPython/issues/1662) regression due to [#​1659](https://togithub.com/gitpython-developers/GitPython/issues/1659) by [@​EliahKagan](https://togithub.com/EliahKagan) in [https://github.com/gitpython-developers/GitPython/pull/1701](https://togithub.com/gitpython-developers/GitPython/pull/1701) - Drop obsolete info on yanking from security policy by [@​EliahKagan](https://togithub.com/EliahKagan) in [https://github.com/gitpython-developers/GitPython/pull/1703](https://togithub.com/gitpython-developers/GitPython/pull/1703) - Have Dependabot offer submodule updates by [@​EliahKagan](https://togithub.com/EliahKagan) in [https://github.com/gitpython-developers/GitPython/pull/1702](https://togithub.com/gitpython-developers/GitPython/pull/1702) - Bump git/ext/gitdb from `49c3178` to `8ec2390` by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/gitpython-developers/GitPython/pull/1704](https://togithub.com/gitpython-developers/GitPython/pull/1704) - Bump git/ext/gitdb from `8ec2390` to `6a22706` by [@​dependabot](https://togithub.com/dependabot) in [https://github.com/gitpython-developers/GitPython/pull/1705](https://togithub.com/gitpython-developers/GitPython/pull/1705) - Update readme for milestone-less releasing by [@​EliahKagan](https://togithub.com/EliahKagan) in [https://github.com/gitpython-developers/GitPython/pull/1707](https://togithub.com/gitpython-developers/GitPython/pull/1707) - Run Cygwin CI workflow commands in login shells by [@​EliahKagan](https://togithub.com/EliahKagan) in [https://github.com/gitpython-developers/GitPython/pull/1709](https://togithub.com/gitpython-developers/GitPython/pull/1709) #### New Contributors - [@​DeflateAwning](https://togithub.com/DeflateAwning) made their first contribution in [https://github.com/gitpython-developers/GitPython/pull/1659](https://togithub.com/gitpython-developers/GitPython/pull/1659) **Full Changelog**: gitpython-developers/GitPython@3.1.37...3.1.38 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/allenporter/flux-local). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4xOS4yIiwidXBkYXRlZEluVmVyIjoiMzcuMTkuMiIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Fixes #1685
Fixes #1686
Changes to
gitmodule codeThis updates the
shellvariable itself, only when it isNone, fromself.USE_SHELL.(That attribute usually gets its value from
Git.USE_SHELLrather than being separately an instance attribute. Butself.USE_SHELLis an idiomatic way to access it. Furthermore, although people probably shouldn't set it on instances, people may expect that this is permitted.)Now:
USE_SHELLis used as a fallback only. Whenshell=Falseis passed,USE_SHELLis no longer consulted. Thusshell=Falsealways keeps a shell from being used, even in the non-default case where theUSE_SHELLattribute has been set toTrue.Popen, because the updated shell variable is used both to produce that message and in thePopencall.New tests
This also (actually, first) adds tests, some to check that the correct value is always passed as
shell=whenGit.executecallsPopen, and others to check that whatever it passes is the same as it claims to be passing in the log. I've made them two separate test methods rather than a test method with multiple assertions, because I think the results of these particular tests are easier to understand that way (even compared to the option of usingself.subTest). But the major shared logic is extracted to a helper function, where its important subtleties are commented. I have verified that the tests fail, in the expected way, before the bugs are fixed, and then pass, and that these are both the case even on native Windows where we don't (yet) have CI jobs.The tests cover the six combinations of valid values of the
shell=argument toGit.execute(None,False, andTrue) andGit.USE_SHELL(FalseandTrue). I usedddtfor this, so there are only two test methods, even though there are six test cases. The tests are simple in principle, with the main source of complexity being the question of how to deal with how the command argument toPopentypically differs, including by type, based on theshellargument, yet the tests must not misbehave, nor induce extraneous misbehavior, even when the code under test has a bug that would ordinarily prevent such agreement.On Python 3.7 only, this adds
mockas a development dependency (in thetestextra, not as a regular dependency).mockis a backport ofunittest.mock, and a couple of my assertions use a handy feature that was introduced to the mock library in Python 3.8. Besides being harder to write without it, I believe it would be less clearly expressed. (Details are commented with the import logic.)While adding the new tests, I also renamed
test_it_executes_git_to_shell_and_returns_resulttotest_it_executes_git_and_returns_result, because that test case does not test that a shell is used to rungit(and a shell is not used in that test).