html repr of xarray object (for the notebook)

[benbovy]

Edit: preview for Dataset and DataArray (pure html/css)

Dataset: https://jsfiddle.net/tay08cn9/4/
DataArray: https://jsfiddle.net/43z4v2wt/9/

---

I started to think a bit more deeply about how could look like a more rich, html-based representation of xarray objects that we would see, e.g., in jupyter notebooks.

Here are some ideas for Dataset: https://jsfiddle.net/9ab4c3tr/35/

Some notes:

• The html repr looks pretty similar than the plain-text repr. I think it's better if they don't differ too much from each other.
• For the sake of consistency, I've stolen some style from pandas.Dataframe repr as it is shown in jupyterlab.
• I tried to emphasize the most important parts of the repr, i.e., the lists of dimensions, coordinates and variables.
• I think it's best if we keep a very lightweight implementation, i.e., pure HTML/CSS (no Javascript). It already allows some interaction like hover effects and collapsible sections. However, I doubt that more fancy stuff (like, e.g., highlighting on hover a specific dimension simultaneously at several places of the repr) would be possible here without Javascript. I have limited skills in this area, though.

It is still, of course, some preliminary thoughts. Any feedback/suggestion is welcome, even opinions about whether an html repr is really needed or not!

[rabernat]

OMG this is so cool!

[shoyer]

OMG indeed this looks fantastic! We should test this on more examples, but I already love it. The underline is a great way to illustrate which variables are dimensions.

I think it's best if we keep a very lightweight implementation, i.e., pure HTML/CSS (no Javascript)

Agreed. Notebook viewers tend to sanitize out JavaScript, especially if originating from an unknown source.

highlighting on hover a specific dimension simultaneously at several places of the repr

This would also be awesome, if possible with pure CSS. I don't know, but I'm going to bug my colleague who does JavaScript visualization.

[shoyer]

OK, so bad news is that it not possible to select elements other than siblings or descendant tags with CSS. So selecting "cousin" tags like a dimension name at multiple locations in the repr is out.

But we could potentially add JavaScript for fancy hover effects. Even if it gets stripped out in many cases (for untrusted notebooks), it should degrade gracefully to the HTML only repr. My main concern would annoying prompts in the notebook interface that ask a user if they want to trust outputs or not.

[darothen]

Wow, great job @benbovy!

With the upcoming move towards Jupyter Lab and a better infrastructure for custom plugins, could this serve as the basis for a "NetCDF Extension" for Jupyter Lab? It would be great if double clicking on a NetCDF file in the JLab file explorer could open up this sort of information, or even a quick and dirty ncview-like plotter.

[benbovy]

Thanks for the feedback!

Here are a few more ideas: https://jsfiddle.net/9ab4c3tr/48/

• main section titles are colored so that these are more detached from the content (not sure I really like it, though),
• subtle shade variations, notably for displaying the first values for each variable,
• drop-downs for displaying attributes per variable if any (collapsed by default).
• hover-effect for dimensions: highlight all variables having the hovered dimension (uses Javascript)

We might use drop-downs to display other useful information as well, such as the type of array (e.g., dask-array, in-memory numpy.array, etc...)

Fancy hover effects would be awesome indeed, although my concern is that too much hover effect would be a source of distraction.

A jupyterlab NetCDF viewer extension would be awesome too! It might also leverage phosphor's datagrid (phosphorjs/phosphor#285) to explore the raw data values.

[rabernat]

Although this could clearly be made more fancy and complicated, I think what you have here would already be a great addition. I love the drop-down attributes (those are missing from the standard repr). Don't hesitate to start a PR! More fanciness could always be added in the future based on user feedback.

…

On Thu, Oct 12, 2017 at 9:01 AM, Benoit Bovy ***@***.***> wrote: Thanks for the feedback! Here is a few more ideas: https://jsfiddle.net/9ab4c3tr/48/ - main sections titles are colored so that these are more detached from the content (not sure I really like it, though), - subtle shade variations, notably for displaying the first values for each variable, - drop-downs for displaying attributes per variable if any (collapsed by default). - hover-effect for dimensions: highlight all variables having the hovered dimension (uses Javascript) We might use drop-downs to display other useful information as well, such as the type of array (e.g., dask-array, in-memory numpy.array, etc...) Fancy hover effects would be awesome indeed, although my concern is that too much hover effect would be a source of distraction. A jupyterlab NetCDF viewer extension would be awesome too! It might also leverage phosphor's datagrid (phosphorjs/phosphor#285 <phosphorjs/phosphor#285>) to explore the raw data values. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1627 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABJFJrhyjDbEVEz5AqUN6l2ppExwE3xiks5srg2MgaJpZM4P2LOr> .

[shoyer]

As long as we're nit-picking style....

main section titles are colored so that these are more detached from the content (not sure I really like it, though),

I slightly prefer the grayscale you had before -- the section titles are already well detached.

subtle shade variations, notably for displaying the first values for each variable,

Love it!

drop-downs for displaying attributes per variable if any (collapsed by default).

It is wonderful to expose this information! I have a slight concern that the + icons could be confusing with the * we use for index variables in the normal repr. I don't have any good ideas for fixing this yet.

hover-effect for dimensions: highlight all variables having the hovered dimension (uses Javascript)

For highlighting, maybe something a little more subtle would work well, either:

1. only highlighting the dimension names below, not the full variables, or
2. only highlight full variables with a matching name, as well as all matching dimensions.

[benbovy]

Before starting a PR on this, I'd like to get the design right (at least the static part) and also to clarify the repr of other xarray objects such as DataArray, Variable, Dataset.coords and Dataset.data_vars.

For Dataset.coords and Dataset.data_vars it would be pretty straightforward (we may simply display the corresponding sections in Dataset html repr).

For DataArray and Variable I have still no clear idea on what would be a good representation for the data values:

• Option A: embed the plain-text representation of the wrapped numpy.array (or dask.array) in a HTML container (https://jsfiddle.net/43z4v2wt/1/). The result is not that bad, although monospace and sans-serif fonts do not mix very well IMO.

• Option B: a html-formatted table for data values as well. That would be tricky for >2d arrays, it has already been discussed in other issues without real consensus.

[benbovy]

have a slight concern that the + icons could be confusing with the * we use for index variables in the normal repr.

Maybe putting the + icon more to the left would do the trick (https://jsfiddle.net/9ab4c3tr/49/).

For highlighting, maybe something a little more subtle would work well

Changing font-weight instead of background-color works better IMO (same link above).

[shoyer]

Option A: embed the plain-text representation of the wrapped numpy.array (or dask.array) in a HTML container (https://jsfiddle.net/43z4v2wt/1/). The result is not that bad, although monospace and sans-serif fonts do not mix very well IMO.

I think this is probably our best option. My main suggestion is it should be possible to click somewhere (maybe on top xarray.DataArray header) to hide/show the data section.

[shoyer]

Maybe putting the + icon more to the left would do the trick (https://jsfiddle.net/9ab4c3tr/49/).

Now it just looks a little out of place :). I liked it better closer to the variable name.

Changing font-weight instead of background-color works better IMO (same link above).

Hmm, I'm not sure. I find text that moves very distracting -- I like changing background colors.

[benbovy]

My main suggestion is it should be possible to click somewhere (maybe on top xarray.DataArray header) to hide/show the data section.

See https://jsfiddle.net/43z4v2wt/3/

I find text that moves very distracting

Indeed, I agree now, it's annoying.

[shoyer]

One more trick we could add for the Dataset repr:

• When hovering over data, show repr for the full array values (as a numpy array) in a box (similar to "title" text, but ideally fixed-width format).

[benbovy]

When hovering over data, show repr for the full array values (as a numpy array) in a box (similar to "title" text, but ideally fixed-width format).

That would be nice, although I guess that it would require javascript.

Alternatively, we can imagine two drop-downs per variable, one for the attributes and one for the full array values (numpy or dask repr). Each would be shown/hidden by two distinct symbols or links, perhaps both located on the right after the preview-values so that we avoid confusion with symbol * in the normal repr.

In the DataArray repr, instead of completely collapsing the data section we may reduce it into a one-line preview of the first values. We can even imagine showing the reduced version by default in cases where the full data section would take too much vertical space.