Feature Request: JUnit stylized output

[rakjennings]

One of the major reasons for using unit tests, is to enable better CI/CD practices. As such, it would be nice if AUnit came with the option of outputting results in a JUnit style so that the output artifacts can be used in things like GitLab test reports and other test reporting tooling.

[bxparks]

Converted to Discussion because I want Feature Requests to be separated from Bugs.

The short answer is that I am not interested in working on this feature, because I don't see much value in it. But if you see value and are willing to do the work, you are welcome to fork the repo and implement it. If the resulting code is easy to understand and maintain, I would be willing to upstream it into the original repo. But if the code is complicated, then I probably would be resistant to accept the PR because I don't want to be responsible for maintaining it.

[bxparks]

I'm curious to know why you think a JUnit-style XML output is worth adding to AUnit. The times that I've seen it useful is when working a large project or product, say with O(100k), O(1M), or O(10M) lines of code, that has dozens or hundreds of developers working on it, and there are so many changelists/PRs coming into the source code, that you need a dashboard to keep track of everything to make sure that the code base does not stay broken for very long.

AUnit is for the Arduino environment. where it appears that almost no one writes unit tests. And only a small fraction of those test writers use continuous builds. And of those, an even smaller fraction those would want unit test dashboards. I wrote EpoxyDuino to allow AUnit to run under GitHub Actions. That has been more than enough for me. If I commit some buggy code, GitHub sends me an email within a few minutes, telling me which test I broke. I go in and fix it. I have trouble seeing how a dashboard helps me in a significant way.

There is one other observation that I would like to make. I have noticed that on large projects with test dashboards, something akin to Goodhard's law can sometimes be observed. Recall that the law says "When a measure becomes a target, it ceases to be a good measure". In this case, the test dashboard is the measure, and since it is highly visible, it becomes the target. This causes engineering resources to be spent pursing pursuing the target (e.g. test coverage percentage, or # of contiguous hours/days that the build was green), instead of pursing pursuing the real target which is the production of high quality code to implement features of the product or project. The build dashboard ceases to be a good measure of the quality of the code base.

Speaking of test coverage percentage, I am of the school of thought that thinks that 100% test coverage target is completely useless and harmful. Depending on the project, 50% test coverage is probably too little, but 100% is definitely too much. A 70% to 80% coverage is probably a good target.

But having said all that, if this is something that is interesting to you, you are totally free to have a go at it.

[rakjennings]

Unfortunately I don't have the time to work on this right now. But the idea is to expose the results of what we already have, unit testing. Its not about making the display a target, or even looking to achieve 100% test coverage. Just exposing what the unit tests I do have results are to a system that collects such reports. A dashboard that collects reports starts helping more when there are either a significant number of people, a significant number of tests, or multiple systems reporting to the same dashboard. My project falls under all three of these scenarios.

[bxparks]

Those may be all good reasons, but I don't work for you or your organization. It sounds like you have access to significant number of people, and significant resources. I don't think it's appropriate to ask for work to be done for free.

[rakjennings]

It is a feature request on an open source project. It seems reasonable submit a ticket for the tool's improvement. A request does not have to be accepted or completed. I do not expect you to do the work. I found a neat OSS tool that solved a problem, had an idea about what could make it better, and submitted a ticket.

You have already stated you do not have desire to do this work. I am not trying to convince you to do the work. I was trying to answer your questions about why this request would be desired. I see now you are not interested in having this conversation or dialog. My apologies.

[bxparks]

I don't understand your statement, "I do not expect you to do the work". You filed a Feature Request. By definition you are requesting me to do the work. You offered no help to do the work. You offered no compensation for the work.

I responded, twice, that I am not interested in this feature, but that you are free to implement it yourself. I suspected that this feature would be only valuable to large projects or organizations. Your response confirmed my suspicions, because I interpreted it as (paraphrased): "This feature is important for my big project with lots of developers, but I'm too cheap or lazy to spend any resources on it, but I want you to implement it for us for free."

This conversation has convinced me that this feature will NEVER be implemented in this project. Because it is seems most useful for corporations working on large projects involving lots of developers, and I have no wish to provide free support for such organizations.