NotebookCellData.mime

[mjbvz]

Problem

The notebooks api has the concept of markup cells. The intent is that these cells are rendered as html by default, unlike code cells which are displayed as editors by default with their rendered output below them.

Currently we only support markdown markup cells. However we'd also like to support other markup languages, including ones that an extension may invent. However there's currently no way to set the mime type of a markup cell, which is needed for a custom renderer to be able to support it

Proposal

Add a new NotebookCellData.mime field. If not explicitly provided, this would default to text/plain (or text/markdown it the cell's language is markdown)

The mime field would only be used when rendering the cell. The existing language field would still be used to determine which type of editor to display for the cell

[DonJayamanne]

Would this allow one to render base64 images?
Assume the text is a base64 encoded image, and we have mime type set to image/png, I'm assuming the renderer could then potentially display that.

I'm assuming this would allow more powerful rendering of content such as rendering images for image/svg+xml, and latex for text/latex

I guess this depends on the definition of markup languages as defined here like to support other markup languages.

Would the rendering architecture/API extensibility be similar to that of renderers?

This means its possible to create something like a transformer (transform the markup text into a visual output). E.g. i could have a cell text containing contents of the XML from a *.drawio file that's renderered as a flowchart.

[mjbvz]

@DonJayamanne Yes, you could do that (although for images, you probably just want to use markdown)

In the future we may explore allowing binary cells as well, but for now the big limitation is that you must have text data. Right now though our focus is very much on the cases you listed, such as latex and more traditional markup languages

[mjbvz]

@rebornix @jrieken I believe we may need to add a mime property to NotebookCell. Right now that entire class is immutable, but we likely want to allow extensions to be able to set the mime type of an existing markup cell. This isn't a problem with language since the extensions can just modify the TextDocument

Let me know if you have any thoughts on this. Is adding a read+write property to NotebookCell acceptable or do we need to find a different way to express this?

[jrieken]

This isn't a problem with language since the extensions can just modify the TextDocument

Extension cannot do that - only the setTextDocumentLanguage function allows you to do that. Both, TextDocument and NotebookDocument and their descendants are immutable.

[mjbvz]

We could also exploring having markup cells use outputs.

Basically this would be a function executeMarkup(markupCell: NotebookCell): NotebookCellOutputItem that maps markup cells to a single output. This output item would then replace the cell when it is not being edited. VS Code could provide a default implementation that handles markdown, but extensions would need to handle more advanced cases

Needs more thought

[rchiodo]

+ 1 on the output idea