After maven action for github

Table of Contents

• Usage
  • Options
• Notes about the implementation
  • TODO
  • History
  • Local development and useage

This contains a script that can be run after a maven build.

This script figures out how many tests ran and how many of them failed. The idea is that maven itself can just ignore test failures, and run to its end.

Failures are also concisely outputted. So, it can also serve as maven oriented, drop in for something like EnricoMi/publish-unit-test-result-action

It just supposes surefire/failsafe testing, and possibly jacoco for coverage.

Usage

Your step to build with maven and report the result might look something like this

steps:
  - uses: actions/checkout@v5
  - uses: actions/setup-java@v5
    with:
      distribution: 'temurin'
      java-version: 17
  - name: Build with Maven
    run: mvn -B -D"maven.test.failure.ignore=true"
  - uses: MMProgrami/after-maven-action@v2
    id: after_maven
    with:
      coverage: true
      determine_version: false
  - name: Use variable
    run: |
      echo "The values are ${{ toJson(steps.after_maven.outputs) }}"
    if: always()

Like this, all tests are run. The after-maven-action will then check how many tests ran and how many of them failed, and will fail the action if there were any test failures. The advantage is that in this way all remaining tests are still run, and it is visible what those are. Also, there won’t be a dip in the coverage if one test happened to fail once.

Options

The script current supports the following options

coverage

If set to true (default to false) then it will also find all jacoco.xml and presents a summary of that.

determine_version

If set to true (which is the default), it will also determine the pom version (the output of mvn -ntp help:evaluate -Dexpression=project.version -q -DforceStdout), and report that. This can be useful information in subsequent steps, but it may slow down things for a second.

Notes about the implementation

This thing will basically search all failsafe/surefire results XMLs and extract the number of tests that ran, failed and errored, and add those up. The results are written in a file '$GITHUB_OUTPUT' which may look something like this:

PROJECT_VERSION=0.19-SNAPSHOT
MAVEN_TESTS_RUN=1626
MAVEN_TESTS_FAILED=8
MAVEN_TESTS_ERROR=7
MAVEN_TESTS_SKIPPED=0

If there are failed or errored tests, the action will exit with a code > 0, which will fail the action.

It will also concisely output which tetsts fails, if there are any.

The script is tested for 'ubuntu-latest', 'windows-latest' and 'macos-latest'.

TODO

• Perhaps it is possible to add some parameters, e.g. to optionally fail on test skips, to fail on too few tests run, or to fail on errored tests only, or whatever one may want to do.

History

Originally this script was using bash/xsltproc, and was (and is) used in dedicated maven docker image, which we use in gitlab pipelines. To make it useable as a github action to I started with just installing the necessary tools and run the same script. This makes it a bit slow as a GitHub action. So starting from v2 it is purely Node.js, just based on the same XSLTs (using SaxonJS to execute them)

Local development and useage

sudo apt-get install nodejs npm
python3 -m venv ~/venvs/npm
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
nvm install 20

make clean install

(main,7)~/github/mihxil/math$ node ~/github/mmprogrami/after-maven-action/dist/index.js
PROJECT_VERSION=0.19-SNAPSHOT
MAVEN_TESTS_RUN=4825
MAVEN_TESTS_FAILED=7
MAVEN_TESTS_ERROR=7
MAVEN_TESTS_SKIPPED=8

mihxil-physics failure: reciprocal org.meeuw.physics.PhysicalNumberTest

mihxil-algebra failure: fromString org.meeuw.test.math.abstractalgebra.complex.ComplexNumberTest
mihxil-algebra failure: times org.meeuw.test.math.abstractalgebra.complex.ComplexNumberTest

mihxil-math-test failure: reciprocal org.meeuw.math.abstractalgebra.reals.RealFieldTest
mihxil-math-test failure: inverse org.meeuw.math.abstractalgebra.reals.RealFieldTest

mihxil-math failure: zeroWithUncertainty org.meeuw.test.math.text.UncertainDoubleFormatTest

mihxil-statistics error: powNegative1 group org.meeuw.math.statistics.StatisticalDoubleTest
mihxil-statistics error: powNegative1 group org.meeuw.math.statistics.StatisticalDoubleTest
mihxil-statistics error: algebraicBinaryOperators org.meeuw.math.statistics.StatisticalDoubleTest
mihxil-statistics error: reciprocal org.meeuw.math.statistics.StatisticalDoubleTest
mihxil-statistics error: division org.meeuw.math.statistics.StatisticalDoubleTest
mihxil-statistics error: algebraicUnaryOperators org.meeuw.math.statistics.StatisticalDoubleTest
mihxil-statistics error: inverse org.meeuw.math.statistics.StatisticalDoubleTest
mihxil-statistics failure: anotherZero org.meeuw.math.statistics.StatisticalDoubleTest

Some (7) tests had errors. Exit 1.