Partest should print test number/test file before running the test so that e. g. hangs can be diagnosed immediately

[scabug]

I think it would be better to have the test status at the right, so that the line with the test number and the test file can be printed before the test is executed. When the test is finished, the result is added at the right side.

E. g.

& starting 1347 tests in run
   1 - run/absoverride.scala                             ok
   2 - run/amp.scala                                     ok 
   3 - run/adding-growing-set.scala                      

In the above example, test 3 would be running.

In cases where a test hangs, it is extremely annoying to not immediately see on which test partest is stuck.
It would also make test failures more intuitive, because the e. g. stacktrace would come after and not before the test number/file name/test status line.

See scala/scala#2418 (comment)

[scabug]

Imported From: https://issues.scala-lang.org/browse/SI-7416?orig=1
Reporter: @soc
Affected Versions: 2.11.0-M2

[scabug]

@paulp said:
In the example above, I guess you are running with one thread? What's that take, 5-6 hours? Because if you're not running one test at a time, you're not going to find it so easy to produce this output.

[scabug]

@soc said:
I had that in mind, but if som-snytt has some Swing UI for it, I think it should at least be possible with that interface.
Making it work in the console is most likely a lot more complicated, I agree.

[scabug]

@som-snytt said:
It's not so easy.

The other side-effect is that you're waiting for the long-runner to say ok while other tests complete, then it spews all the oks until the next running test. It's more fun to see notifications as they occur (out of submit order).

For this use case, maybe you really want per-test timeout.

As recently noted, it would be nice if it could tell you when space or time requirements for a test appear to deviate. (Even: this test used to complete in five seconds and now it takes twenty. Maybe a Jenkins-like indicator to accompany, if not stormy weather then the rumblings of distant thunder.)

[scabug]

@soc said (edited on Apr 25, 2013 2:44:34 PM UTC):

It's not so easy.

I agree and I wouldn't have created an issue if I hadn't been told to do so. :-)

[som-snytt]

currently

ok    7 - run/Course-2002-02.scala                [duration 5.11s]

Maybe sbt now offers a test progress indicator.

I came looking for a ticket about how a pos test does not show -Vdebug -Vlog output, even under --verbose.

[SethTisue]

ironically, I could have used this 15 minutes ago. (I thought adding --verbose would do the trick but it didn't.)

The culprit turned out to be a stray call to System.exit in a forked test, causing an EOFException in partest's innards, but I had to bisect (repeatedly throw away half the tests) to identify that culprit 🤷

(Regardless, I support the decision not to keep the ticket open.)

[som-snytt]

I considered transferring to vulpix.

A related feature was to start longer-running tests earlier; otherwise, a test group waits while the last test finishes.

"Contributions welcome" at vulpix.

Edit: in case this gets transferred to vulpix, I'd like to see N windows or panes showing the concurrent output of N workers. Either in-terminal or browser.

[som-snytt]

Oh, also ironically, while squashing the PR for 7415, I noticed I had a branch for 7416, which was this ticket before it was transferred.

It's a small bug world.