Add functions to View, Display, and Code (formerly Print) to a stream

[markuspf]

This pull request is part of an attempt (and a first step) at cleaning up GAP's output path, it is part of an attempt to provide libgap functionality as well as a cleaner Jupyter interface within the GAP codebase.

I propose to first add the operations ViewObjToStream, CodeObjToStream, and DisplayObjToStream. The added code is to be understood as a first suggestion, is fully backwards compatible, and does not change any of GAP's behaviour yet.

This change should make it possible to implement ViewObj, ViewString, PrintObj, PrintString, Display, and DisplayString in terms of the above operations, removing the need to implement two variants of every function (one for the direct output and one for the String variant).

(I am aware of the existence of PrintTo and AppendTo)

[ChrisJefferson]

Are we intending that these should have the magic 'indent/unindent' markers?

Assuming we still like them, they probably should, as they are easy to strip out when don't want them (outputting the stream to a string for example), if we do it in the kernel the time won't be measurable.

[ChrisJefferson]

Also want to give an ordering (I assume each will delegate in some order, need to define clearly what that order is).

[codecov-io]

Codecov Report

Merging #895 into master will increase coverage by 1.12%.
The diff coverage is 46.42%.

@@            Coverage Diff             @@
##           master     #895      +/-   ##
==========================================
+ Coverage   61.69%   62.82%   +1.12%     
==========================================
  Files         970      970              
  Lines      295646   295320     -326     
  Branches    13057    13055       -2     
==========================================
+ Hits       182395   185521    +3126     
+ Misses     110612   106997    -3615     
- Partials     2639     2802     +163

Impacted Files	Coverage Δ
lib/show.gi	0% <0%> (ø)
lib/record.gi	49.05% <49.05%> (ø)
src/range.c	87.39% <0%> (-0.22%)	⬇️
src/sysfiles.c	30.46% <0%> (-0.15%)	⬇️
src/vars.h	97.05% <0%> (-0.09%)	⬇️
src/hpc/traverse.c	78.59% <0%> (-0.09%)	⬇️
hpcgap/lib/type1.g
src/intrprtr.c	70.9% <0%> (+0.17%)	⬆️
... and 14 more

[markuspf]

I am not even sure whether I want delegation at this level, or just fallback to some reasonably uninformative "<object>". A higher level function could try to find the "best fit" if it is installed. This would help prevent "rabbithole" delegations that are not very nice to debug.

Is there a good argument to delegate? Does python delegate between repr and str for instance?

There is of course also the question whether both View and Display are needed. I'd say yes, because a brief output is usually better for REPL, but one might want something more verbose (which makes me consider calling DisplayObjToStream VerboseViewObjToStream)

[markuspf]

As for the markers: At least for "formatted output" the markers are of course useful. I somewhat would wish that View only ever outputs one line (so no need for indent), Display and Code could be using indent markers and linebreak hints, but we can use the underlying stream to strip them out if we want to.

The goal should in my opinion be to do less implicit magic, but be able to do it explicitly if wanted.

[fingolfin]

As for the markers: It is of course possible to strip them out when generating a string; we'd then want a special string stream "subtype" which does that for us, I guess (to minimize the overhead)?
Of course it still would cause a little overhead, but if we do it right, I'd hope it'd be negligible..

But for completeness, here is another alternative: instead of "printing" marker strings into streams, use special "marker objects", and perhaps a special "marker aware" print function... I'll try to illustrate this with a hypothetical example. Consider this, from the GAP library:

  Print("PartialPerm(\>\> ", DomainOfPartialPerm(f), "\<, \>",
     ImageListOfPartialPerm(f), "\<\< )");

It might become something like this:

  PrintWithMarkers(stream,
    "PartialPerm(", Indent,
        Indent, ", DomainOfPartialPerm(f), Unindent, ", ",
        Indent, ImageListOfPartialPerm(f), Unindent, Unindent, " )");

or perhaps

  PrintWithMarkers(stream,
    "PartialPerm(", Indent(2), ", DomainOfPartialPerm(f), Indent(-1), ", ",
        Indent(1), ImageListOfPartialPerm(f), Indent(-1), " )");

Here, the function PrintWithMarkers would ignore the indent markers when printing into a string stream, but handle them when printing to a terminal.

The whole thing is inspired by C++'s cout << "Hello, world" << endl;.

Yet another alternative would be to have a separate (un)indent function, like so:

  PrintTo(stream, "PartialPerm(");
  ChangeIndent(stream, 2);
  PrintTo(stream, ", DomainOfPartialPerm(f)")
  ChangeIndent(stream, -1);
  PrintTo(stream, ", ");
  ChangeIndent(stream, 1);
  PrintTo(stream, ImageListOfPartialPerm(f))
  ChangeIndent(stream, -1);
  PrintTo(stream, " )");

This is perhaps easier to implement, but more cumbersome, and IMHO harder to read.

[fingolfin]

The main problem in my eyes with the whole ViewString / DisplayString etc. project in my view is that there never was a coherent and realistic transition strategy. And indeed, we never transitioned, and trying to is very painful, for various reasons I won't go into much (but one of them is the weird back and forth delegation between them)

In addition, the semantics of Display/ViewObj/Print resp. DisplayString/ViewString/String are inconsistent and confusing, making things even more painful.

To resolve this, using streams is IMHO the right approach: It is less cumbersome and fragile than trying to capture the output of a print operation to the screen. And it has the potential of avoiding the performance problem that is caused if one first prints into a string and then outputs the string, which can become serious if one prints a lot of data (but note that I say it has the potential -- it all depends on how much overhead the streams cause by themselves. I'd be more comfortable with this work if it included some benchmarks that studied the performance different between, say, ViewObj, Print(ViewString(x)) and ViewStream.

Moreover, there should be a clear plan on what to do with this. At the very least, a few "real" methods for ViewStream and DisplayStream should be implemented. That is, for some object/type, the existing DisplayString/Display/ViewString/ViewObj methods should be replaced by their stream counterparts, to demonstrate what is necessary for this. One possible strategy would be to still provide ViewObj/Display methods which just delegate to their stream counterparts. To simplify that, we could add a helper function, say InstallViewObjMethodUsingViewStream(type). Then, when "everything" (or at least "enough") is converted to use streams, one could possibly replace those with a single set of default ViewObj/Display methods -- but that's in the far future.

Another question: What's up with Print and PrintString? They are curiously absent from this.

Finally, we really would have to document how the various methods are supposed to interact -- the GAP manual has already something on this, which would have to be extended for the stream functions.

[markuspf]

About the Print and PrintString: I left it out deliberately, and as the title says want to use a different name (Code) to capture what is currently semi-documented as what Print is supposed to do: Provide a code-representation of the object that can be pasted back into a GAP session.

The reason for this is that Print is just a mess and prints whatever the programmer at the time thought would be nice to print (like names, other representations), and my feeling was that a "clean slate" for the code representation would be a good idea (tm) (famous last words, etc). Also my association for Print would always be "put this string on screen".

I'll rebase and make a fresh go at this, maybe on the train next week (though my train time is dangerously overcommitted as well again).

[fingolfin]

Don't start too many projects, it's fine to leave this sitting for another day. It's probably better to leave this for the relase after 4.9 anyway.

I'd rather see us resolve open issues, impove the package build system etc

[markuspf]

Just as a small remark: I added CodeObjStream and ViewObjStream for records to this PR, but did not replace the ViewObj, PrintObj, DisplayObj yet, since I am prodding about indentation handling at the moment (this is currently muddled in setting some global print state, but could as @fingolfin suggested be folded into a stream that supports indentation handling and just indentation of appropriate depth after every newline that is printed)

I have also been experimenting with pretty printers in the pull request #1132 borrowing from Wadlers "prettier printers" paper, but that is very much a work-in-progress.

[fingolfin]

Might as well remove the reference to compat3c.g.

Also, I think this renaming and splitting of record.g is of independent interest, so we could merge it before the rest of this PR (which would reduce the diff in this PR, making it easier to review).

[fingolfin]

Change this to record.gi?