You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Test results [](https://semaphore.semaphoreci.com/projects/test-results-cli)
2
+
3
+
Semaphore collects XML test reports and uses them to provide insight into your pipelines.
4
+
5
+
With test reports, you enable your team to get an effective and consistent view of your CI/CD test suite across different test frameworks and stages in a CI/CD workflow. You get a clear failure report for each executed pipeline. Failures are extracted and highlighted, while the rest of the suite is available for analysis.
6
+
7
+
The test-results command-line interface (CLI) is a tool that helps you compile and process [JUnit XML](https://github.com/windyroad/JUnit-Schema/blob/master/JUnit.xsd) files. The output of the test results CLI is a report in JSON format.
8
+
9
+
This CLI is distributed as a part of the [Semaphore toolbox](https://github.com/semaphoreci/toolbox), and it is available in all Semaphore jobs.
10
+
11
+
The main purpose of the CLI is to:
12
+
13
+
- compile and publish JUnit XML files into a JSON report
14
+
- merge multiple JSON reports into a single summary report
15
+
16
+
> [!NOTE]
17
+
>
18
+
> Starting from version `0.7.0`, test reports are compressed using gzip to optimize storage and handling. To maintain backward compatibility with existing systems and processes, the compressed files will continue to use the `.json` file extension. This approach ensures seamless integration with tools and workflows that expect files in this format.
19
+
>
20
+
> However, please be aware that while these files retain the `.json` extension, they are in a compressed format and will need to be decompressed using gzip-compatible tools before they can be read as standard JSON.
21
+
>
22
+
> For users who prefer to work with uncompressed reports or for systems that require non-compressed files, we've introduced a `--no-compress` option. This can be used to generate and upload test reports in the traditional, uncompressed JSON format.
23
+
24
+
## Compiling and publishing JUnit XML files
25
+
26
+
Given your JUnit XML report is named `results.xml` you can run the following command to generate a report:
27
+
28
+
```bash
29
+
test-results publish results.xml
30
+
```
31
+
32
+
The above command parses the content of the `results.xml` file, and publishes the results to Semaphore.
33
+
34
+
While parsing the content, the CLI tries to find the best parser for your result type. The following test runners have a dedicated parser:
35
+
36
+
- exunit
37
+
- golang
38
+
- mocha
39
+
- rspec
40
+
- phpunit
41
+
42
+
If a dedicated parser is not found, the CLI will parse the file using a generic parser. The generic parser uses [JUnit XML Schema](https://github.com/windyroad/JUnit-Schema/blob/master/JUnit.xsd) definition to extract data from the report.
43
+
44
+
The parser can be selected manually by using the `--parser` option.
45
+
46
+
```bash
47
+
test-results publish --parser exunit results.xml
48
+
```
49
+
50
+
The name of the generated report is based on the selected parser. If you want to overwrite this you can use the `--name` option:
The generated tests in a report will sometimes contain a prefix in the name. For example `Elixir.MyTest`. If you want to remove the `Elixir` prefix from the test names you can use `--suite-prefix` option:
If your job generates multiple reports: `integration.xml`, `unit.xml` you can use this command to merge and publish them
65
+
66
+
```bash
67
+
test-results publish integration.xml unit.xml
68
+
```
69
+
70
+
In addition, each report is published separately to artifact storage as a `junit-<index>.xml`. `<index>` is a number starting from 0 that corresponds to the order of the report passed to the command line.
71
+
72
+
## Merging multiple JSON reports into a single summary report
73
+
74
+
If you have multiple jobs in your pipeline that generate test results, you can merge them into a single report with the following command
75
+
76
+
```bash
77
+
test-results gen-pipeline-report
78
+
```
79
+
80
+
The above command assumes you are running it in a semaphore pipeline. As it uses `SEMAPHORE_PIPELINE_ID` environment variable to identify the pipeline and fetch the job level reports.
81
+
82
+
## Where are test reports stored?
83
+
84
+
The test results CLI uses the [Semaphore Artifact Storage](https://docs.semaphoreci.com/essentials/artifacts/) to store the test reports:
85
+
86
+
- the `test-results publish` command stores the report in the `test-results/junit.json` file on a job level
87
+
- the `test-results gen-pipeline-report` command stores the report in the `test-results/${SEMAPHORE_PIPELINE_ID}.json` file on a workflow level
88
+
89
+
Expiration date of the artifacts can be controlled via [Retention Policies](https://docs.semaphoreci.com/essentials/artifacts/#artifact-retention-policies).
90
+
91
+
## Skip uploading raw JUnit XML files
92
+
93
+
By default, `test-results publish` will upload the raw JUnit XML file alongside the JSON report to the artifact storage. This can be disabled with the `--no-raw` option:
94
+
95
+
```bash
96
+
test-results publish --no-raw results.xml
97
+
```
98
+
99
+
## Overwrite existing reports
100
+
101
+
By default `test-results publish` and `test-results gen-pipeline-report` will fail if the report is already present in the artifact storage. This behaviour can be disabled with the `--force` option:
102
+
103
+
```bash
104
+
# Publish the report
105
+
test-results publish results.xml
106
+
107
+
#...
108
+
109
+
# other-results will overwrite results
110
+
test-results publish --force other-results.xml
111
+
```
112
+
113
+
## Using the CLI on a local machine
114
+
115
+
Latest CLI binaries are available [here](https://github.com/semaphoreci/test-results/releases/latest).
0 commit comments