Compatibility problem with Python 3.9?

[mhjacks]

Hello,

I'm the maintainer of git-up in Fedora. Recently we did a mass archive against Python 3.9 and git-up's test started failing. Here's the relevant log:

+ rm -rfv /builddir/build/BUILDROOT/git-up-1.6.1-5.fc33.x86_64/usr/bin/__pycache__
+ /usr/lib/rpm/find-debuginfo.sh -j8 --strict-build-id -m -i --build-id-seed 1.6.1-5.fc33 --unique-debug-suffix -1.6.1-5.fc33.x86_64 --unique-debug-src-base git-up-1.6.1-5.fc33.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 -S debugsourcefiles.list /builddir/build/BUILD/git-up-1.6.1
find: 'debug': No such file or directory
+ /usr/lib/rpm/check-buildroot
+ /usr/lib/rpm/redhat/brp-ldconfig
+ /usr/lib/rpm/brp-compress
+ /usr/lib/rpm/redhat/brp-strip-lto /usr/bin/strip
+ /usr/lib/rpm/brp-strip-static-archive /usr/bin/strip
+ /usr/lib/rpm/redhat/brp-python-bytecompile '' 1 0
Bytecompiling .py files below /builddir/build/BUILDROOT/git-up-1.6.1-5.fc33.x86_64/usr/lib/python3.9 using /usr/bin/python3.9
+ /usr/lib/rpm/brp-python-hardlink
+ /usr/lib/rpm/redhat/brp-mangle-shebangs
Executing(%check): /bin/sh -e /var/tmp/rpm-tmp.YFhh0w
+ umask 022
+ cd /builddir/build/BUILD
+ cd git-up-1.6.1
+ git config --global user.email koji@fedoraproject.org
+ git config --global user.name 'Koji Build System'
+ PYTHONPATH=/builddir/build/BUILDROOT/git-up-1.6.1-5.fc33.x86_64/usr/lib/python3.9/site-packages
+ nosetests-3.9
.S..S...S........................E..S
======================================================================
ERROR: test suite for <module 'PyGitUp.tests.test_up_to_date' from '/builddir/build/BUILD/git-up-1.6.1/PyGitUp/tests/test_up_to_date.py'>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/nose/suite.py", line 209, in run
    self.setUp()
  File "/usr/lib/python3.9/site-packages/nose/suite.py", line 292, in setUp
    self.setupContext(ancestor)
  File "/usr/lib/python3.9/site-packages/nose/suite.py", line 315, in setupContext
    try_run(context, names)
  File "/usr/lib/python3.9/site-packages/nose/util.py", line 471, in try_run
    return func()
  File "/builddir/build/BUILD/git-up-1.6.1/PyGitUp/tests/test_up_to_date.py", line 32, in setup
    repo.remotes.origin.pull()
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 811, in pull
    res = self._get_fetch_info_from_stderr(proc, progress)
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 708, in _get_fetch_info_from_stderr
    output.extend(FetchInfo._from_line(self.repo, err_line, fetch_line)
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 708, in <genexpr>
    output.extend(FetchInfo._from_line(self.repo, err_line, fetch_line)
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 292, in _from_line
    raise ValueError("Failed to parse line: %r" % line)
ValueError: Failed to parse line: '  git config pull.rebase false  # merge (the default strategy)'
-------------------- >> begin captured logging << --------------------
git.cmd: DEBUG: Popen(['git', 'init'], cwd=/tmp/PyGitUp.ufk366oy/master.up-to-date, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['git', 'cat-file', '--batch-check'], cwd=/tmp/PyGitUp.ufk366oy/master.up-to-date, universal_newlines=False, shell=None, istream=<valid stream>)
git.cmd: DEBUG: Popen(['git', 'cat-file', '--batch'], cwd=/tmp/PyGitUp.ufk366oy/master.up-to-date, universal_newlines=False, shell=None, istream=<valid stream>)
git.cmd: DEBUG: Popen(['git', 'checkout', '-b', 'initial'], cwd=/tmp/PyGitUp.ufk366oy/master.up-to-date, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['git', 'checkout', '-b', 'up-to-date'], cwd=/tmp/PyGitUp.ufk366oy/master.up-to-date, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['git', 'clone', '-b', 'up-to-date', '-v', '/tmp/PyGitUp.ufk366oy/master.up-to-date/.git', '/tmp/PyGitUp.ufk366oy/up-to-date'], cwd=/tmp/PyGitUp.ufk366oy/master.up-to-date, universal_newlines=True, shell=None, istream=None)
git.repo.base: DEBUG: Cmd(['git', 'clone', '-b', 'up-to-date', '-v', '/tmp/PyGitUp.ufk366oy/master.up-to-date/.git', '/tmp/PyGitUp.ufk366oy/up-to-date'])'s unused stdout: 
git.cmd: DEBUG: Popen(['git', 'pull', '-v', 'origin'], cwd=/tmp/PyGitUp.ufk366oy/up-to-date, universal_newlines=True, shell=None, istream=None)
git.remote: DEBUG: Fetch head lines do not match lines provided via progress information
length of progress lines 6 should be equal to lines in FETCH_HEAD file 3
Will ignore extra progress lines or fetch head lines.
git.remote: DEBUG: info lines: ['  git config pull.rebase false  # merge (the default strategy)', '  git config pull.rebase true   # rebase', '  git config pull.ff only       # fast-forward only', '   eb2d67d..6adb676  up-to-date -> origin/up-to-date', ' = [up to date]      initial    -> origin/initial', ' = [up to date]      master     -> origin/master']
git.remote: DEBUG: head info : ["6adb676af33e7f95cf91d1fc3841adbbbcbc97f7\t\tbranch 'up-to-date' of /tmp/PyGitUp.ufk366oy/master.up-to-date/\n", "eb2d67d81794f11739aa098c22b41e129a2ce70d\tnot-for-merge\tbranch 'initial' of /tmp/PyGitUp.ufk366oy/master.up-to-date/\n", "eb2d67d81794f11739aa098c22b41e129a2ce70d\tnot-for-merge\tbranch 'master' of /tmp/PyGitUp.ufk366oy/master.up-to-date/\n"]
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 36 tests in 3.301s

FAILED (SKIP=4, errors=1)

I'm not sure why there would be a different hash here. Do you think this might be due to a change in Python 3.9?

[msiemens]

Interesting. I suspect this is more due to a Git update than caused by the Python upgrade. I use Python 3.8 locally and tests started to fail for me too. I'll look into this…

[msiemens]

Seems like this is related to gitpython-developers/GitPython#1014 which in turn is caused by git/git@d18c950 which was published in Git v2.27.0. The issue has been fixed in GitPython v3.1.3 but we're using GitPython 2.x. Updating to GitPython 3.x means dropping Python 2 support. So I'll publish a release that drops Python 2 support in order to fix this issue.

[mhjacks]

Thank you! That will work well for us as we don't support Python 2 in Fedora anymore.

[msiemens]

Should be fixed with v2.0.0 🙃

[mhjacks]

I don't see setup.py in the new release tarball?

[msiemens]

Oh, weird. I forgot to publish the release to PyPI. The release there should contain the setup.py. I switched the build to Poetry which generates the setup.py when building the release artifacts.

[mhjacks]

Ah, that makes sense. Thanks! I was unaware that Poetry did that. I'll work on packaging 2.0.0 for Fedora.

[mhjacks]

OK, I'm working on packaging, and I've run a super-minor test problem (init.py in tests/ doesn't seem to have import os and the test suite complains about that). But there's also this:

% nosetests-3

..............................E...
======================================================================
ERROR: test suite for <module 'PyGitUp.tests.test_up_to_date' from '/home/mjackson/git-up-2.0.0/PyGitUp/tests/test_up_to_date.py'>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/nose/suite.py", line 209, in run
    self.setUp()
  File "/usr/lib/python3.9/site-packages/nose/suite.py", line 292, in setUp
    self.setupContext(ancestor)
  File "/usr/lib/python3.9/site-packages/nose/suite.py", line 315, in setupContext
    try_run(context, names)
  File "/usr/lib/python3.9/site-packages/nose/util.py", line 471, in try_run
    return func()
  File "/home/mjackson/git-up-2.0.0/PyGitUp/tests/test_up_to_date.py", line 30, in setup
    repo.remotes.origin.pull()
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 811, in pull
    res = self._get_fetch_info_from_stderr(proc, progress)
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 708, in _get_fetch_info_from_stderr
    output.extend(FetchInfo._from_line(self.repo, err_line, fetch_line)
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 708, in <genexpr>
    output.extend(FetchInfo._from_line(self.repo, err_line, fetch_line)
  File "/usr/lib/python3.9/site-packages/git/remote.py", line 292, in _from_line
    raise ValueError("Failed to parse line: %r" % line)
ValueError: Failed to parse line: '  git config pull.rebase false  # merge (the default strategy)'
-------------------- >> begin captured logging << --------------------
git.cmd: DEBUG: Popen(['git', 'init'], cwd=/tmp/PyGitUp.y1q5d8ld/master.up-to-date, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['git', 'cat-file', '--batch-check'], cwd=/tmp/PyGitUp.y1q5d8ld/master.up-to-date, universal_newlines=False, shell=None, istream=<valid stream>)
git.cmd: DEBUG: Popen(['git', 'cat-file', '--batch'], cwd=/tmp/PyGitUp.y1q5d8ld/master.up-to-date, universal_newlines=False, shell=None, istream=<valid stream>)
git.cmd: DEBUG: Popen(['git', 'checkout', '-b', 'initial'], cwd=/tmp/PyGitUp.y1q5d8ld/master.up-to-date, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['git', 'checkout', '-b', 'up-to-date'], cwd=/tmp/PyGitUp.y1q5d8ld/master.up-to-date, universal_newlines=False, shell=None, istream=None)
git.cmd: DEBUG: Popen(['git', 'clone', '-b', 'up-to-date', '-v', '/tmp/PyGitUp.y1q5d8ld/master.up-to-date/.git', '/tmp/PyGitUp.y1q5d8ld/up-to-date'], cwd=/tmp/PyGitUp.y1q5d8ld/master.up-to-date, universal_newlines=True, shell=None, istream=None)
git.repo.base: DEBUG: Cmd(['git', 'clone', '-b', 'up-to-date', '-v', '/tmp/PyGitUp.y1q5d8ld/master.up-to-date/.git', '/tmp/PyGitUp.y1q5d8ld/up-to-date'])'s unused stdout: 
git.cmd: DEBUG: Popen(['git', 'pull', '-v', 'origin'], cwd=/tmp/PyGitUp.y1q5d8ld/up-to-date, universal_newlines=True, shell=None, istream=None)
git.remote: DEBUG: Fetch head lines do not match lines provided via progress information
length of progress lines 6 should be equal to lines in FETCH_HEAD file 3
Will ignore extra progress lines or fetch head lines.
git.remote: DEBUG: info lines: ['  git config pull.rebase false  # merge (the default strategy)', '  git config pull.rebase true   # rebase', '  git config pull.ff only       # fast-forward only', '   4ee720c..78fba12  up-to-date -> origin/up-to-date', ' = [up to date]      initial    -> origin/initial', ' = [up to date]      master     -> origin/master']
git.remote: DEBUG: head info : ["78fba12789eaf1eb43fbf45239b0efdb553a062b\t\tbranch 'up-to-date' of /tmp/PyGitUp.y1q5d8ld/master.up-to-date/\n", "4ee720cca1268fb98eaef7f6cb4832d009703729\tnot-for-merge\tbranch 'initial' of /tmp/PyGitUp.y1q5d8ld/master.up-to-date/\n", "4ee720cca1268fb98eaef7f6cb4832d009703729\tnot-for-merge\tbranch 'master' of /tmp/PyGitUp.y1q5d8ld/master.up-to-date/\n"]
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 33 tests in 4.027s

Is this another case of "the output changed" or is it something more significant?

[msiemens]

That's strange. Seems like this is the exact issue that you had at the start. Could you post the output of running python -c 'import git; print(git.__version__)' in your build environment where PyGitUp is installed?

[mhjacks]

% python -c 'import git; print(git.__version__)'
3.0.9

Which is interesting, because the package claims to be 3.1.0:
python3-GitPython-3.1.0-3.fc33.noarch

Clearly the package is installing 3.0.9 though:
% rpm -ql python3-GitPython-3.1.0-3.fc33.noarch
/usr/lib/python3.9/site-packages/GitPython-3.0.9-py3.9.egg-info
/usr/lib/python3.9/site-packages/GitPython-3.0.9-py3.9.egg-info/PKG-INFO
/usr/lib/python3.9/site-packages/GitPython-3.0.9-py3.9.egg-info/SOURCES.txt
/usr/lib/python3.9/site-packages/GitPython-3.0.9-py3.9.egg-info/dependency_links.txt
/usr/lib/python3.9/site-packages/GitPython-3.0.9-py3.9.egg-info/not-zip-safe
/usr/lib/python3.9/site-packages/GitPython-3.0.9-py3.9.egg-info/requires.txt
/usr/lib/python3.9/site-packages/GitPython-3.0.9-py3.9.egg-info/top_level.txt

[mhjacks]

It appears we're past the initially reported issue, so I'm ready to close this (Fedora mislabeling a package version isn't really your problem). I see you also loosened the dependency on colorama. Thanks! Would you please consider adding the import os to the test suite as well?

[msiemens]

Would you please consider adding the import os to the test suite as well?

Where is it missing? As far as I can tell, the tests run without errors

[mhjacks]

When I run nosetests on f32

I get this error:

% nosetests
..................................E
======================================================================
ERROR: test suite for <module 'PyGitUp.tests' from '/home/mjackson/git-up-2.0.1/PyGitUp/tests/__init__.py'>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/nose/suite.py", line 228, in run
    self.tearDown()
  File "/usr/lib/python3.8/site-packages/nose/suite.py", line 351, in tearDown
    self.teardownContext(ancestor)
  File "/usr/lib/python3.8/site-packages/nose/suite.py", line 367, in teardownContext
    try_run(context, names)
  File "/usr/lib/python3.8/site-packages/nose/util.py", line 471, in try_run
    return func()
  File "/home/mjackson/git-up-2.0.1/PyGitUp/tests/__init__.py", line 43, in teardown
    os.chdir(join(basepath, '..'))
NameError: name 'os' is not defined

----------------------------------------------------------------------
Ran 34 tests in 4.596s

FAILED (errors=1)

I do not see an import os in this file:
https://github.com/msiemens/PyGitUp/blob/master/PyGitUp/tests/__init__.py so I suspect that's why I'm seeing this error. I could be wrong, though.

[msiemens]

Funny! Since PyGitUp 2.0 I switched to PyTest for running the tests and there everything runs fine. But running with nosetests throws an exception as in your example. I'll publish a post-release with a fix

[msiemens]

This should be fixed now with v2.0.1.post1