Test Runner: delegate arbitrary output formatting to each reporter

[MoLow]

What is the problem this feature will solve?

The current implementation of the test runner (--test) is very TAP oriented,
so we have decided (#43525 (comment), #45618) to treat any arbitrary output as a test diagnostic.

I do agree this decision makes sense when using TAP as the reporter, but I think we can provide better DX for console logs or other arbitrary output when using other reporters.

What is the feature you are proposing to solve the problem?

add a new event emitted by TestsStream, perhaps test:stdout or test:unknown (or whatever) so each reporter can decide how to treat unknown output.
I think console.log("test") should show the same when running node --test-reporter spec test.js and node --test --test-reporter spec test.js (probably also the same colors etc, but that probably justifies a separate issue)

What alternatives have you considered?

No response

[MoLow]

CC @nodejs/test_runner

[MoLow]

one thing that is TAP oriented and might be harder to change is the fact we direct any output on stderr to stdout - but that might make sense in the context of a test runner.

[HinataKah0]

Hi @MoLow ,

Is anyone from test runner team working on it? Or is it open for anyone to try?
It seems that it's still being discussed 🤔

[MoLow]

@HinataKah0 I first want to gather feedback regarding this issue, then it makes sense to only work on it after #47867 lands

[MoLow]

I have discussed this with @cjihrig and he says it makes sense to him.
@HinataKah0 do you want to implement this?

[HinataKah0]

I'll be happy to try 😄
Will send a PR so we can discuss as well (I hope I don't misunderstand anything)

[cjihrig]

one thing that is TAP oriented and might be harder to change is the fact we direct any output on stderr to stdout - but that might make sense in the context of a test runner.

I think it makes sense for the test runner to collect both stdout and stderr from the child processes and direct them to the same place. For example, if the reporter destination is a file, having stdout and stderr both sent to the file is helpful for reviewing the output at a later time. If stderr is not sent to the reporter destination, there would be no way of recovering it later.