Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

junit-xml output attribute can be too big to handle with -vv #12223

Open
Redoubts opened this issue Apr 17, 2024 · 0 comments
Open

junit-xml output attribute can be too big to handle with -vv #12223

Redoubts opened this issue Apr 17, 2024 · 0 comments
Labels
plugin: junitxml related to the junitxml builtin plugin type: performance performance or memory problem/improvement

Comments

@Redoubts
Copy link

Suppose you have a test that may fail with a very large diff; say:

def test_xyz() -> None:
    assert ["x"] * 10_000 == ["x\n"] * 10_000

With normal verbosity (0), the junit-xml output would look like this:

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
 <testsuite errors="0" failures="1" hostname="..." name="pytest" skipped="0" tests="1" time="1.172" timestamp="...">
  <testcase classname="test_file" file="test_file.py" line="0" name="test_xyz" time="0.767">
   <failure message="AssertionError: assert ['x', 'x', 'x...'x', 'x', ...] == ['x\n', 'x\n'...', 'x\n', ...]
  At index 0 diff: 'x' != 'x\n'
  Use -v to get more diff">
    def test_xyz() -&gt; None:
&gt;       assert ["x"] * 10_000 == ["x\n"] * 10_000
E       AssertionError: assert ['x', 'x', 'x...'x', 'x', ...] == ['x\n', 'x\n'...', 'x\n', ...]
E         At index 0 diff: 'x' != 'x\n'
E         Use -v to get more diff

test_file.py:2: AssertionError
   </failure>
  </testcase>
 </testsuite>
</testsuites>

noting the <failure message='...'> tag's message and body both have a Use -v to get more diff.

If you do supply a -vv, then both the body of the <failure> tag, AND the content of the message attribute can be very large. The latter can be problematic for parsers, and projects like BeautifulSoup and lxml will fail to parse this correctly (truncating or crashing, respectively).

Maybe this is a limitation of the junit schema, but is there anything acceptable that can be done to reduce only the content of the message attribute? I think it's more expected and "fine" for the body of a tag to be really long, but an attribute less so.

applies to pytest 8.1.1

@Zac-HD Zac-HD added type: performance performance or memory problem/improvement plugin: junitxml related to the junitxml builtin plugin labels Apr 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
plugin: junitxml related to the junitxml builtin plugin type: performance performance or memory problem/improvement
Projects
None yet
Development

No branches or pull requests

2 participants