Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: check result with test_510 - test likely needs fixing #6813

Closed
wants to merge 1 commit into from

Conversation

blueyed
Copy link
Contributor

@blueyed blueyed commented Feb 24, 2020

Note that the output changed with 9e26203 (#3623), so the test should likely
be fixed.

Ref: #519 (- haven't looked into what the issue is about).
Might also be #510 ? (mixed names)

Example script was added in c081c5e, changed in ea90605, currently outputs:

Previous output:

['============================= test session starts ==============================',
 'platform linux -- Python 3.8.1, pytest-5.3.5.dev567+gfba7d992a.d20200224, py-1.8.2.dev3+g2b6bd292, pluggy-0.13.1',
 'rootdir: ~',
 'collected 8 items',
 '',
 'issue_519.py ........E                                                   [100%]',
 '',
 '==================================== ERRORS ====================================',
 '_________________ ERROR at teardown of test_two[arg1v2-arg2v2] _________________',
 '',
 '    @pytest.fixture(scope="session")',
 '    def checked_order():',
 '        order = []',
 '    ',
 '        yield order',
 '        pprint.pprint(order)',
 '>       assert order == [',
 '            ("testing/example_scripts/issue_519.py", "fix1", "arg1v1"),',
 '            ("test_one[arg1v1-arg2v1]", "fix2", "arg2v1"),',
 '            ("test_two[arg1v1-arg2v1]", "fix2", "arg2v1"),',
 '            ("test_one[arg1v1-arg2v2]", "fix2", "arg2v2"),',
 '            ("test_two[arg1v1-arg2v2]", "fix2", "arg2v2"),',
 '            ("testing/example_scripts/issue_519.py", "fix1", "arg1v2"),',
 '            ("test_one[arg1v2-arg2v1]", "fix2", "arg2v1"),',
 '            ("test_two[arg1v2-arg2v1]", "fix2", "arg2v1"),',
 '            ("test_one[arg1v2-arg2v2]", "fix2", "arg2v2"),',
 '            ("test_two[arg1v2-arg2v2]", "fix2", "arg2v2"),',
 '        ]',
 "E       AssertionError: assert [('issue_519....arg1v2'), ...] == [('testing/ex...arg1v2'), ...]",
 'E         At index 0 diff:',
 "E         ('issue_519.py', 'fix1', 'arg1v1') !=",
 "E         ('testing/example_scripts/issue_519.py', 'fix1', 'arg1v1')",
 'E         Use -v to get the full diff',
 '',
 "issue_519.py:20: AssertionError: assert [('issue_519....arg1v2'), ...] == [('testing/ex...arg1v2'), ...]...",
 '--------------------------- Captured stdout teardown ---------------------------',
 "[('issue_519.py', 'fix1', 'arg1v1'),",
 " ('test_one[arg1v1-arg2v1]', 'fix2', 'arg2v1'),",
 " ('test_two[arg1v1-arg2v1]', 'fix2', 'arg2v1'),",
 " ('test_one[arg1v1-arg2v2]', 'fix2', 'arg2v2'),",
 " ('test_two[arg1v1-arg2v2]', 'fix2', 'arg2v2'),",
 " ('issue_519.py', 'fix1', 'arg1v2'),",
 " ('test_one[arg1v2-arg2v1]', 'fix2', 'arg2v1'),",
 " ('test_two[arg1v2-arg2v1]', 'fix2', 'arg2v1'),",
 " ('test_one[arg1v2-arg2v2]', 'fix2', 'arg2v2'),",
 " ('test_two[arg1v2-arg2v2]', 'fix2', 'arg2v2')]",

Note that the output changed with 9e26203, so the test should likely
be fixed.

Ref: pytest-dev#519 (- haven't looked into what the issue is about).

Example script was added in c081c5e, changed in ea90605, currently outputs:

Previous output:
```
['============================= test session starts ==============================',
 'platform linux -- Python 3.8.1, pytest-5.3.5.dev567+gfba7d992a.d20200224, py-1.8.2.dev3+g2b6bd292, pluggy-0.13.1',
 'rootdir: ~',
 'collected 8 items',
 '',
 'issue_519.py ........E                                                   [100%]',
 '',
 '==================================== ERRORS ====================================',
 '_________________ ERROR at teardown of test_two[arg1v2-arg2v2] _________________',
 '',
 '    @pytest.fixture(scope="session")',
 '    def checked_order():',
 '        order = []',
 '    ',
 '        yield order',
 '        pprint.pprint(order)',
 '>       assert order == [',
 '            ("testing/example_scripts/issue_519.py", "fix1", "arg1v1"),',
 '            ("test_one[arg1v1-arg2v1]", "fix2", "arg2v1"),',
 '            ("test_two[arg1v1-arg2v1]", "fix2", "arg2v1"),',
 '            ("test_one[arg1v1-arg2v2]", "fix2", "arg2v2"),',
 '            ("test_two[arg1v1-arg2v2]", "fix2", "arg2v2"),',
 '            ("testing/example_scripts/issue_519.py", "fix1", "arg1v2"),',
 '            ("test_one[arg1v2-arg2v1]", "fix2", "arg2v1"),',
 '            ("test_two[arg1v2-arg2v1]", "fix2", "arg2v1"),',
 '            ("test_one[arg1v2-arg2v2]", "fix2", "arg2v2"),',
 '            ("test_two[arg1v2-arg2v2]", "fix2", "arg2v2"),',
 '        ]',
 "E       AssertionError: assert [('issue_519....arg1v2'), ...] == [('testing/ex...arg1v2'), ...]",
 'E         At index 0 diff:',
 "E         ('issue_519.py', 'fix1', 'arg1v1') !=",
 "E         ('testing/example_scripts/issue_519.py', 'fix1', 'arg1v1')",
 'E         Use -v to get the full diff',
 '',
 "issue_519.py:20: AssertionError: assert [('issue_519....arg1v2'), ...] == [('testing/ex...arg1v2'), ...]...",
 '--------------------------- Captured stdout teardown ---------------------------',
 "[('issue_519.py', 'fix1', 'arg1v1'),",
 " ('test_one[arg1v1-arg2v1]', 'fix2', 'arg2v1'),",
 " ('test_two[arg1v1-arg2v1]', 'fix2', 'arg2v1'),",
 " ('test_one[arg1v1-arg2v2]', 'fix2', 'arg2v2'),",
 " ('test_two[arg1v1-arg2v2]', 'fix2', 'arg2v2'),",
 " ('issue_519.py', 'fix1', 'arg1v2'),",
 " ('test_one[arg1v2-arg2v1]', 'fix2', 'arg2v1'),",
 " ('test_two[arg1v2-arg2v1]', 'fix2', 'arg2v1'),",
 " ('test_one[arg1v2-arg2v2]', 'fix2', 'arg2v2'),",
 " ('test_two[arg1v2-arg2v2]', 'fix2', 'arg2v2')]",
```
@blueyed blueyed added type: bug problem that needs to be addressed type: selftests a problem in the tests of pytest labels Feb 24, 2020
blueyed added a commit to blueyed/pytest that referenced this pull request Feb 24, 2020
Output is not being looked at, and it changed/broke recently apparently.

Ref: pytest-dev#6813
@blueyed
Copy link
Contributor Author

blueyed commented Feb 24, 2020

Turned into an issue, since I do not plan to work on this.
#6814

@blueyed blueyed closed this Feb 24, 2020
@blueyed blueyed deleted the test_issue519 branch February 24, 2020 23:15
blueyed added a commit to blueyed/pytest that referenced this pull request Feb 24, 2020
Output is not being looked at, and it changed/broke recently apparently.

Ref: pytest-dev#6813
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug problem that needs to be addressed type: selftests a problem in the tests of pytest
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant