Test: add option variant showProgress:=some

[fingolfin]

This PR adds a new experimental showProgress mode to TestDirectory and Test, which shows the current line of the .tst file being processed. This is useful to gauge the progress of a longer running test, and can thus help identify which parts of the tests are particularly slow. Moreover, if a test completely hangs (as currently happens with some testinstall tests for HPC-GAP), with this PR it is trivial to know in which line the test is stuck.

Of course the same can also already now be achieved via showProgress:=true -- but that prints tons of stuff, which makes it difficult to find actual diffs. I.e. too much noise drowns out the signal.

The crucial part of the patch is this line: Print("\r# line ", pos[i], "\c"); It returns the output to the start of the current terminal line, then prints the line number; the \c at the end ensures the output buffer is flushed, i.e. the output text is visible. This way, each new line number overwrites the last line number (instead of printing a new line).

Finally, at the end we use Print("\r \c\r"); to overwrite the line number output.

The PR also enables this feature as new default. The one drawback is that this causes a test failure for the test of the Test() function itself! The reason for this is that while visually, the output of Test did not change (at least when printed to a terminal), it actually did change: The test diff "sees" the extra line numbers and complains about a diff.

To avoid this, I followed @ChrisJefferson's suggestion and turned the Test test from Example to Log, so they are not executed.

[ChrisJefferson]

I'd like to see this merged.

Easier option -- change the manual examples for Test from Example to Log so they are not executed -- we certainly test Test enough in other ways (and could do a proper Test test if someone wanted to check the more obscure options).

[fingolfin]

This PR is "ready" now: I added documentation and made sure the tests pass, following @ChrisJefferson's suggestion.

[codecov]

Codecov Report

Merging #1230 into master will decrease coverage by <.01%.
The diff coverage is 70%.

@@            Coverage Diff             @@
##           master    #1230      +/-   ##
==========================================
- Coverage   61.77%   61.76%   -0.01%     
==========================================
  Files        1034     1034              
  Lines      356536   356534       -2     
  Branches    14283    14281       -2     
==========================================
- Hits       220234   220213      -21     
- Misses     132651   132670      +19     
  Partials     3651     3651

Impacted Files	Coverage Δ
lib/test.gi	44.94% <70%> (-5.16%)	⬇️
lib/files.gd	61.29% <0%> (-1.08%)	⬇️
src/hpc/traverse.c	79.45% <0%> (-0.39%)	⬇️
src/hpc/threadapi.c	31.14% <0%> (-0.3%)	⬇️
src/funcs.c	64.34% <0%> (ø)	⬆️
src/scanner.c	70.24% <0%> (+0.17%)	⬆️
src/system.c	54.76% <0%> (+0.68%)	⬆️
src/objset.c	39.88% <0%> (+1.12%)	⬆️

[hulpke]

Excellent idea.

[olexandr-konovalov]

Thanks, I have checked out this PR and run testinstall - very nice, makes watching how tests are passing much more entertaining! Nicely done - files in dev/log are not polluted by the output that is shown only on screen.

[olexandr-konovalov]

It seems that the documentation is slightly incorrect: if set to "some", then GAP shows the number of the current line of the test being processed, not the line itself.

[fingolfin]

OK, I se how "shows the current line" is ambiguous (for me, that actually does convey "line number", but I agree it's not clear enough, will rephrase).

[olexandr-konovalov]

I would make the comment more clear, e.g. Show some progress (the number of the current line of the test being processed) or a bit shorter - but more informative then just "some progress".

[fingolfin]

But this is an internal comment, summarizing the defaults -- if people need more clarity on what these options mean, they should look at the manual entry

[fingolfin]

in this sense, I am tempted to remove all the comments there :-)

[olexandr-konovalov]

Ok - let's remove (except lines 541-542) and ease maintenance tasks :)

[fingolfin]

How does that ease maintenance? Do I really need to rebase again to get rid of some comments? Also, which of them? Why are those in 541/542 more helpful? Also, somebody put those comments there, so if I remove them now, that person might disagree with the change... Huh

[olexandr-konovalov]

How does that ease maintenance?

because otherwise there is a higher chance that someone will update documentation in .gd file but not comments in .gi

Do I really need to rebase again to get rid of some comments? Also, which of them?
Why are those in 541/542 more helpful?

I would leave 541-542 in the form

## TestDirectory(<files> [, <options> ])
## <files>: A directory (or filename) or list of filenames and directories
## <options>: Optional record of options

because they seem more stable (unlike options which may change again) and may be helpful for anyone looking at the code to quickly see the meaning of the arguments. The lines below may be deleted, indeed.

Also, somebody put those comments there, so if I remove them now, that person might disagree with the change... Huh

I've approved your changes anyway, so you may wish to go ahead and merge this as it is, if you're reluctant to remove them. TestDirectory is by @ChrisJefferson IIRC, so maybe he would comment briefly on what he thinks...