Robot Framework's XUnit Output Modifier (xom)

Simple implementation for Robot Framework XUnit output modifier. The modifier implementation produces identical XUnit output file as Robot Framework's XUnit output does, but the modifier xom.py is much easier to custom various needs of XUnit output. Several requests discussed on Robot Framework community are pinpointed to the code allowing fast forward implementation like:

• Plural root node <testsuites> when multiple testsuites in a test run.
• Separate root node attributes for <testsuites> element.
• Testsuite attribute hostname.
• XUnit timestamp to UTC time.
• Local time UTC offset to testsuite's property element.
• Testcase attributes file and lineno pointing to the test source file and line number.
• Timestamp modification.

This work is derived from Robot Framework's XUnitFileWriter: https://github.com/robotframework/robotframework/blob/master/src/robot/reporting/xunitwriter.py

Eventually, some modification work introduced here could be promoted to Robot Framework itself.

Contribution

All contributions are the most welcome. Feel free to file a bug report or enhancement request. Improve code with PRs, create tests or make the documentation more fluent.

Simple Installation

Clone this repository or just download the xom.py file. Place the xom.py file to your PYTHONPATH or use --pythonpath specifier for Robot Framework test run. See examples below. Examples assume xom.py is located to very same folder than .robot files.

Example Usage

Naturally using xom requires Robot Framework installed.

General usage with robot or rebot:

--prerebotmodifier [module_name].[class_name]:[output_filename]

Get xunit.xml output file from the modifier:

robot --pythonpath . --prerebotmodifier xom.XUnitOut:xunit.xml test.robot

Same with rebot:

rebot --pythonpath . --prerebotmodifier xom.XUnitOut:xunit.xml output.xml

NOTE: Do not use same filename for the modifier and Robot Framework's XUnit output. That won't work, because Robot Framework's XUnitWriter (specified with -x) overwrites the target file:

robot --pythonpath . --prerebotmodifier xom.XUnitOut:xunit.xml -x xunit.xml test.robot

This works fine:

robot --pythonpath . --prerebotmodifier xom.XUnitOut:xcustom.xml -x xdefault.xml test.robot

Possible Modifications

Example code modification points are denoted with # * comment lines in the code.

Configuration Flags

Configuration flags are set to False by default.

• Root node is generated to <testsuites> when multiple suites in a test run.

  ROOT_NODE_PLURAL = True

• Robot Framework uses local time for test suite's timestamp. Use this flag to set timestamp from local system time to UTC time.

  XUNIT_UTC_TIME_IN_USE = True

• Report local time offset to UTC time in seconds to Suite's property element. Offset is negative to east from GMT and positive to west from GMT.

  REPORT_UTC_TIME_OFFSET = True

Root Node

Root node could have plural form <testsuites>. You may modify <testsuites> root element's attributes:

    if ROOT_NODE_PLURAL and suite.parent is None and suite.suites:
        stats = suite.statistics  # Accessing property only once.
        attrs = {
            'name': suite.name,
            'time': self._time_as_seconds(suite.elapsedtime),
            'tests': f'{stats.total}',
            'failures': f'{stats.failed}',
            'disabled': f'{stats.skipped}', # If you feel that skipped tests maps to disabled.
            'errors': '0'
        }
        self._writer.start('testsuites', attrs)

See also JUnit team's xsd reference.

Testsuite Attribute hostname

To get hostname within testsuite element you may use following reference implementation:

import platform

def start_suite(self, suite):
    stats = suite.statistics  # Accessing property only once.
    attrs = {'name':       suite.name,
            'tests':       f'{stats.total}',
            # No meaningful data available from suite for `errors`.
            'errors':      '0',
            'failures':    f'{stats.failed}',
            'skipped':     f'{stats.skipped}',
            'time':        self._time_as_seconds(suite.elapsedtime),
            'timestamp':   self._starttime_to_isoformat(suite.starttime),
            'hostname':    platform.uname().node,
            }
    self._writer.start('testsuite', attrs)

Testcase Attribute file And lineno

Testcase attributes file as testcase's source filename and lineno as line number of testcase in the source file.

def visit_test(self, test):
    attrs = {'classname': test.parent.longname,
                'name': test.name,
                'time': self._time_as_seconds(test.elapsedtime),
                'file': test.source,
                'lineno': f'{test.lineno}',
                }
    self._writer.start('testcase', attrs)

Method _starttime_to_isoformat to custom timestamp.

Alternate XUnit output's timestamps to your favor.

Full-Blown Example Output

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="Test &amp; Test2" tests="4" errors="0" failures="1" skipped="1" time="0.037" timestamp="2022-06-09T17:57:05000" hostname="osstest-desktop-1">
    <testsuite name="Test" tests="3" errors="0" failures="1" skipped="1" time="0.013" timestamp="2022-06-09T17:57:05000" hostname="osstest-desktop-1">
        <testcase classname="Test &amp; Test2.Test" name="Metadata Test" time="0.003" file="/home/osstest/demo/test.robot" lineno="2">
        </testcase>
        <testcase classname="Test &amp; Test2.Test" name="Test Failing" time="0.004" file="/home/osstest/demo/test.robot" lineno="8">
            <failure message="This is failing case." type="AssertionError"/>
        </testcase>
        <testcase classname="Test &amp; Test2.Test" name="Test Skip" time="0.002" file="/home/osstest/demo/test.robot" lineno="12">
            <skipped message="This is skipped case." type="SkipExecution"/>
        </testcase>
        <properties>
            <property name="Documentation" value="Test suite to demonstrate XUnit modification features."/>
            <property name="metaname" value="metavalue"/>
            <property name="metaname2" value="metavalue2"/>
            <property name="utc-offset" value="-10800"/>
        </properties>
    </testsuite>
    <testsuite name="Test2" tests="1" errors="0" failures="0" skipped="0" time="0.004" timestamp="2022-06-09T17:57:05000" hostname="osstest-desktop-1">
        <testcase classname="Test &amp; Test2.Test2" name="Just One Thing To Test" time="0.002" file="/home/osstest/demo/test2.robot" lineno="2">
        </testcase>
        <properties>
            <property name="utc-offset" value="-10800"/>
        </properties>
    </testsuite>
    <properties>
        <property name="utc-offset" value="-10800"/>
    </properties>
</testsuites>

References

Robot Framework's prerebotmodifier

JUnit team's xds

Robot Framework API

Robot Framework's Visitor Model

Robot Framework's Testsuite Model