Show XFail reason as part of JUnitXML message field

[samueljsb]

Fixes #4907

[codecov]

Codecov Report

Merging #5087 into features will increase coverage by <.01%.
The diff coverage is 100%.

@@             Coverage Diff              @@
##           features    #5087      +/-   ##
============================================
+ Coverage     96.09%   96.09%   +<.01%     
============================================
  Files           115      115              
  Lines         25894    25907      +13     
  Branches       2560     2561       +1     
============================================
+ Hits          24883    24896      +13     
  Misses          704      704              
  Partials        307      307

Impacted Files	Coverage Δ
testing/test_junitxml.py	97.84% <100%> (+0.03%)	⬆️
src/_pytest/junitxml.py	95.25% <100%> (+0.04%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 97cd5f0...a37d1df. Read the comment docs.

[blueyed]

LGTM, but that's not my area of expertise.

[nicoddemus]

Hi @samueljsb, thanks for the PR!

Current master generates:

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="0" name="pytest" skipped="1" tests="2" time="0.096">
    <testcase classname="foo" file="foo.py" line="3" name="test" time="0.001">
        <skipped message="expected test failure">xfail for some reason</skipped>
    </testcase>
    <testcase classname="foo" file="foo.py" line="8" name="test_ok" time="0.001"></testcase>
</testsuite>

Your PR generates:

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="0" name="pytest" skipped="1" tests="2" time="0.074">
    <testcase classname="foo" file="foo.py" line="3" name="test" time="0.002">
        <skipped message="xfail for some reason" type="pytest.xfail"></skipped>
    </testcase>
    <testcase classname="foo" file="foo.py" line="8" name="test_ok" time="0.000"></testcase>
</testsuite>

The type property added to the skipped item does conform with the latest junit specification:

    <xs:element name="skipped">
        <xs:complexType mixed="true">
            <xs:attribute name="type" type="xs:string"/>
            <xs:attribute name="message" type="xs:string"/>
        </xs:complexType>
    </xs:element>

So I think we are good here. 👍 (created #5095 as a follow up btw)

[nicoddemus]

Hmm although this can be considered a bug-fix, it does change how we produce the XML and might break custom scripts which process the file and expect what pytest already generates.

I think we should rebase this to features and change the change log as a new feature. What do you think @blueyed?

[blueyed]

Yes, "features" sounds better.

@samueldg
You can rebase it, and force-push here, then change the base branch from the "Edit" button next to the PR title. Also rename the changelog file then, please.

[samueljsb]

I'll sort that out now 👍

[nicoddemus]

Thanks a lot @samueljsb!