Skip to content

pytest 7.0.0rc1 and 6.2.5 both fail test_errors_in_xfail_skip_expressions on Python 3.10.1 #9413

@hroncok

Description

@hroncok

In Fedora, when we updated Python from 3.10.0 to 3.10.1, the following test started to fail when we build pytest 6.2.5:

____________________ test_errors_in_xfail_skip_expressions _____________________
pytester = <Pytester PosixPath('/tmp/pytest-of-mockbuild/pytest-0/test_errors_in_xfail_skip_expressions0')>
    def test_errors_in_xfail_skip_expressions(pytester: Pytester) -> None:
        pytester.makepyfile(
            """
            import pytest
            @pytest.mark.skipif("asd")
            def test_nameerror():
                pass
            @pytest.mark.xfail("syntax error")
            def test_syntax():
                pass
    
            def test_func():
                pass
        """
        )
        result = pytester.runpytest()
        markline = "                ^"
        pypy_version_info = getattr(sys, "pypy_version_info", None)
        if pypy_version_info is not None and pypy_version_info < (6,):
            markline = markline[5:]
        elif sys.version_info[:2] >= (3, 10):
            markline = markline[11:]
        elif sys.version_info >= (3, 8) or hasattr(sys, "pypy_version_info"):
            markline = markline[4:]
    
        if sys.version_info[:2] >= (3, 10):
            expected = [
                "*ERROR*test_nameerror*",
                "*asd*",
                "",
                "During handling of the above exception, another exception occurred:",
            ]
        else:
            expected = [
                "*ERROR*test_nameerror*",
            ]
    
        expected += [
            "*evaluating*skipif*condition*",
            "*asd*",
            "*ERROR*test_syntax*",
            "*evaluating*xfail*condition*",
            "    syntax error",
            markline,
            "SyntaxError: invalid syntax",
            "*1 pass*2 errors*",
        ]
>       result.stdout.fnmatch_lines(expected)
E       Failed: nomatch: '*ERROR*test_nameerror*'
E           and: '============================= test session starts =============================='
E           and: 'platform linux -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-1.0.0'
E           and: 'rootdir: /tmp/pytest-of-mockbuild/pytest-0/test_errors_in_xfail_skip_expressions0'
E           and: 'collected 3 items'
E           and: ''
E           and: 'test_errors_in_xfail_skip_expressions.py EE.                             [100%]'
E           and: ''
E           and: '==================================== ERRORS ===================================='
E       fnmatch: '*ERROR*test_nameerror*'
E          with: '_______________________ ERROR at setup of test_nameerror _______________________'
E       fnmatch: '*asd*'
E          with: "name 'asd' is not defined"
E       exact match: ''
E       exact match: 'During handling of the above exception, another exception occurred:'
E       fnmatch: '*evaluating*skipif*condition*'
E          with: "Error evaluating 'skipif' condition"
E       fnmatch: '*asd*'
E          with: '    asd'
E       nomatch: '*ERROR*test_syntax*'
E           and: "NameError: name 'asd' is not defined"
E       fnmatch: '*ERROR*test_syntax*'
E          with: '________________________ ERROR at setup of test_syntax _________________________'
E       nomatch: '*evaluating*xfail*condition*'
E           and: 'invalid syntax (<xfail condition>, line 1)'
E           and: ''
E           and: 'During handling of the above exception, another exception occurred:'
E       fnmatch: '*evaluating*xfail*condition*'
E          with: "Error evaluating 'xfail' condition"
E       exact match: '    syntax error'
E       nomatch: '     ^'
E           and: '            ^'
E           and: 'SyntaxError: invalid syntax'
E           and: '=========================== short test summary info ============================'
E           and: 'ERROR test_errors_in_xfail_skip_expressions.py::test_nameerror'
E           and: 'ERROR test_errors_in_xfail_skip_expressions.py::test_syntax'
E           and: '========================= 1 passed, 2 errors in 0.01s =========================='
E       remains unmatched: '     ^'
/builddir/build/BUILD/pytest-6.2.5/testing/test_skipping.py:1156: Failed
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /tmp/pytest-of-mockbuild/pytest-0/test_errors_in_xfail_skip_expressions0
collected 3 items
test_errors_in_xfail_skip_expressions.py EE.                             [100%]
==================================== ERRORS ====================================
_______________________ ERROR at setup of test_nameerror _______________________
name 'asd' is not defined
During handling of the above exception, another exception occurred:
Error evaluating 'skipif' condition
    asd
NameError: name 'asd' is not defined
________________________ ERROR at setup of test_syntax _________________________
invalid syntax (<xfail condition>, line 1)
During handling of the above exception, another exception occurred:
Error evaluating 'xfail' condition
    syntax error
            ^
SyntaxError: invalid syntax
=========================== short test summary info ============================
ERROR test_errors_in_xfail_skip_expressions.py::test_nameerror
ERROR test_errors_in_xfail_skip_expressions.py::test_syntax
========================= 1 passed, 2 errors in 0.01s ==========================
...
====== 1 failed, 3035 passed, 11 skipped, 10 xfailed in 278.15s (0:04:38) ======

I get the same failure when I attempt to update to 7.0.0rc1.

I suppose this is related to

elif sys.version_info[:2] >= (3, 10):
markline = markline[11:]

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions