pytest-subtest 0.14.0 fails with Python 3.9 and 3.10

[xqt]

The new pytest-subtest release 0.14.0 fails with Python 3.9 and 3.10. Here is a traceback we got for tests:

15:10:46     def _addSkip(self: TestCaseFunction, testcase: TestCase, reason: str) -> None:
15:10:46         from unittest.case import _SubTest  # type: ignore[attr-defined]
15:10:46     
15:10:46         if isinstance(testcase, _SubTest):
15:10:46             self._originaladdSkip(testcase, reason)  # type: ignore[attr-defined]
15:10:46             if self._excinfo is not None:
15:10:46                 exc_info = self._excinfo[-1]
15:10:46                 self.addSubTest(testcase.test_case, testcase, exc_info)  # type: ignore[attr-defined]
15:10:46         else:
15:10:46             # For python < 3.11: the non-subtest skips have to be added by `_originaladdSkip` only after all subtest
15:10:46             # failures are processed by `_addSubTest`.
15:10:46             if sys.version_info < (3, 11):
15:10:46                 subtest_errors = [
15:10:46                     x
15:10:46 >                   for x, y in self.instance._outcome.errors
15:10:46                     if isinstance(x, _SubTest) and y is not None
15:10:46                 ]
15:10:46 E               AttributeError: 'NoneType' object has no attribute 'errors'

See for example: https://integration.wikimedia.org/ci/job/pywikibot-core-tox-fasttest-py39/1805/console

[nicoddemus]

Thanks for the report @xqt!

I will take a look soon, meanwhile cc'ing @ydshieh in case he wants to investigate.

[ydshieh]

Hi @xqt Thank you for reporting.

Locally, when I check, I can see self.instance._outcome is unittest.case._Outcome object and its definition (python 3.9 in my environment) is

class _Outcome(object):
    def __init__(self, result=None):
        self.expecting_failure = False
        self.result = result
        self.result_supports_subtests = hasattr(result, "addSubTest")
        self.success = True
        self.skipped = []
        self.expectedFailure = None
        self.errors = []

which have errors.

My python 3.9 is 3.9.20. Also my env with python 3.10.13 also works. Could you provide your full python version.
From the link you provided, I saw it is 3.9.18, but when I checked with a new local env. with python 3.9.18, it still works.

[ydshieh]

Ah I see here self.instance._outcome is None! Let me check

[ydshieh]

I can reproduce now:

@unittest.skip("bonbon")
class T(unittest.TestCase):
    def test_foo(self):
        assert 1 == 2

It's related to the block

envs\py39\Lib\unittest\case.py

    def run(self, result=None):
        ....
        try:
            breakpoint()
            testMethod = getattr(self, self._testMethodName)
            if (getattr(self.__class__, "__unittest_skip__", False) or
                getattr(testMethod, "__unittest_skip__", False)):
                # If the class or method was skipped.
                skip_why = (getattr(self.__class__, '__unittest_skip_why__', '')
                            or getattr(testMethod, '__unittest_skip_why__', ''))
                self._addSkip(result, self, skip_why)
                return result

I will open a PR to fix it.

[ydshieh]

A fix is opened #174 🙏 @xqt it would be great if you can also check against that PR 🙏

[loadams]

I hit this as well, and I was able to confirm the branch in the PR fixes the issue for me.

[nicoddemus]

0.14.1 released, thanks everyone for reporting and @ydshieh for implementing the fix!