Can/should we take repr seriously?

[tkf]

Currently, repr in Julia is maybe evaluatable or parseable or (often) just a string representation only consumable by humans. However, it could be much useful if it is guaranteed to be evaluatable. For example, you can find a lot of examples in base/loading.jl which is used as simple IPC mechanism:

julia/base/loading.jl

Lines 1148 to 1151 in 9a8b2fd

	code = """
	append!(empty!(Base.DEPOT_PATH), $(repr(map(abspath, DEPOT_PATH))))
	append!(empty!(Base.DL_LOAD_PATH), $(repr(map(abspath, DL_LOAD_PATH))))
	"""

In case of base/loading.jl it is OK to use repr because the types used there are known to have evaluatable repr. However, some basic types like PkgId cannot be used in this way:

julia> repr(Base.PkgId(InteractiveUtils))
"InteractiveUtils [b77e0a4c-d291-57a0-90e8-8db25a27a240]"

Another concrete example I have in mind where repr can be useful is to create a plot "spec" using DataVoyager.jl and dump it into a runnable Julia script. Currently, using it needs a rather convoluted invocation like show(IOContext(stdout, :compact=>false), "text/plain", spec) because repr is not a reliable entry point for evaluatable code.

We have Serialization module for serializing complex Julia objects. Unfortunately, it cannot be used for communicating with different Julia versions. For complex serializations it is probably a good idea to use something like JSON instead of Julia code. However, I think the above examples are good enough motivation to have repr for simple cases.

It was suggested to add a flag like parseable (e.g., #33178 (comment), #30683 (comment)) via the IOContext mechanism. I have a concern about this direction because it is practically impossible to enforce parsability this way (a brief discussion with @JeffBezanson in #30757 (comment)). I think the overloading API should be clear that the implementer has to make it evaluatable. This can be done by introducing overloading API such as show(::IO, ::MIME"application/julia", obj) or repr(::IO, obj).

Questions:

• Do we need an API to produce Julia code that is (very likely to be) evaluatable? (100% guarantee is impossible when using different versions of libraries and/or Julia)
• How should it be implemented? Based on IOContext? New entry point (e.g., show(::IO, ::MIME"application/julia", obj))? Something else?

[JeffBezanson]

I have a concern about this direction because it is practically impossible to enforce parsability this way

I don't really see why --- telling people to handle the flag doesn't seem any worse than telling people to add a particular method.

[vtjnash]

Aside: eval-uatable or parse-able can be totally unrelated things, so it's not necessarily meaningful to say it would be both. There's also just many things that don't have a eval representation, but can be serialized or printed.

[JeffBezanson]

A big part of the problem seems to be that it's hard to characterize how array/dataframe elements should be printed in the REPL. They don't need to be parseable, and we want to use a "nice"/non-parseable representation when possible, but we also want to print type info like the f0 suffix, and we want to quote strings. It's a mix of requirements that's hard to associate with a single function like show or print that should have a simple description.

[tkf]

I don't really see why --- telling people to handle the flag doesn't seem any worse than telling people to add a particular method.

I believe relying on people to not forget something is a bad idea. People forgetting to define HasEltype is a good example. I think overloading API should help people recognizing that they must construct (at least) parseable representation.

It's a mix of requirements that's hard to associate with a single function like show or print that should have a simple description.

Isn't it an argument favoring adding an API for repr implementation? It would decouple repr from display and print.

[JeffBezanson]

OK, adding repr(::IO, x) would be pretty simple and worth considering.

One thing I want to discuss more is, what are the use cases for non-parseable show? Are there any types where we want print, show, and repr to all be different? I'm skeptical. Again, it seems most of the complexity is in how we want array elements to look, rather than in needing many different kinds of printing per se. For example, given these definitions:

print: pretty, concise, doesn't need to be parseable
show: parseable

I think the desired array element printing is very close to:

default: print
numbers: print if type is implied by eltype, otherwise show
strings: show

Any examples where that would go wrong? And are there any other contexts like this?

[tkf]

One thing I want to discuss more is, what are the use cases for non-parseable show?

The only reason I can think of is backward compatibility. It would be great if show(::IO, x) were not used for array's text/plain show and it is used only for repr. I'm proposing a new entry point since it sounds difficult for entire Julia community to switch to it. It probably is easier to introduce repr(::IO, x) and then deprecate show(::IO, x) while switching to Julia 2.0.

I understand that co-existence of similar APIs is very unsatisfactory. But addition of a new API is the only option I can think of to clean up show infrastructure and make repr work while preserving backward compatibility (which of course can be due to my lack of imagination).

I think the desired array element printing is very close to:

default: print
numbers: print if type is implied by eltype, otherwise show
strings: show

I suggest to not include this logic inside array printing. Rather, I think array printer should just set compact and typeinfo for text/plain show and then the above logic should be implemented inside text/plain show of each type. This is because:

• It's usable for other printing of collections (especially table-like data structures).
• It avoids confusion that show can be used to pretty-print element values. (OK, maybe this is weak, given that not many people implement numbers)

[tkf]

I think array printer should just set compact and typeinfo for text/plain show

It also should set displaysize to handle something like a vector of vectors, so that inner array can switch to printing like [1, 2, 3] when the height is 1.

[oxinabox]

I think we should be using MIME types to control this?
I thought we already were.

[vtjnash]

Yes, we use MIME for that. We briefly used an IOContext flag for it (multiline instead of displaysize[2] == 1, but otherwise basically the same). It was a violation of the purpose of IOContext, and thus did badly—search the old issues for this. We could potentially test for both options (MIME(text/plain) + displaysize[2] == 1 is a valid test combo in 3-arg mime-show), but the caller would need to be prepared to handle multiline output then (since IOContext values are only advisory).

[tkf]

I think #34387 is a big step for fixing the situation (thanks to fchorney!). Ref: #30901 (comment)

But not everything is resolved since show(io::IO, ::MIME"text/plain", X::AbstractArray) still does not set IOContext(io, :compact => true) always

julia/base/arrayshow.jl

Lines 329 to 332 in 1d918dd

	# 1) compute new IOContext
	if !haskey(io, :compact) && length(axes(X, 2)) > 1
	io = IOContext(io, :compact => true)
	end

That is to say, 3-arg show for element types still cannot detect if it is called by the show of the container in a reliable manner.

@JeffBezanson You suggested to keep this in #34387 (comment) (just for now?). Is there a plan to improve this?

Also, it would be nice if fallback to 2-arg show

julia/base/arrayshow.jl

Lines 108 to 111 in 1d918dd

	# If the output contains line breaks, try 2-arg show instead.
	if occursin('\n', sx)
	sx = sprint(show, x, context=io, sizehint=0)
	end

can be removed in a future version of Julia. If 2-arg show is meant to be used for repr, we should expect this to generate larger output on average. Furthermore, the fallback to 2-arg show motivates users to overload it to customize how it is printed inside container. This often contradicts with how repr works.