Admonition text blocks support

[milmazz]

What is an admonition text block?

From the AsciiDoc documentation an admonition:

There are certain statements that you may want to draw attention to by taking
them out of the content’s flow and labeling them with a priority. These are
called admonitions. It’s rendered style is determined by the assigned label
(i.e., value)

Also, probably some of you have read the blog post announcing Distillery
2.0, in that blog post Paul mentioned some documentation improvements in
Distillery, specifically:

The docs have been significantly revamped using MkDocs. They are much easier
to navigate now, are more readable by extracting tips and warnings into
asides/callouts, and are fully searchable as well!

Behind the scenes MkDocs uses Python-MarkDown, which parses MarkDown
and support admonitions via an extension.

So, I think it would be really nice to have support for something like this in ExDoc.

About the syntax

The Admonition extension from Python-MarkDown follows almost the same syntax
as in reStructuredText:

.. note:: This is a note admonition.
   This is the second line of the first paragraph.

   - The note contains all indented body elements
     following.
   - It includes this bullet list.

But, they use !!! instead of ..:

!!! note
    You should note that the title will be automatically capitalized.

I also have seen these callouts when I have used AsciiDoc, but the AsciiDoc syntax depends on the use case.

For a single paragraph it is like this:

WARNING: Wolpertingers are known to nest in server racks.
Enter at your own risk.

In this case the label must be uppercase and immediately followed by a colon. But the syntax is a little more complex for a block of text:

[IMPORTANT]
.Feeding the Werewolves
====
While werewolves are hardy community members, keep in mind the following dietary concerns:

. They are allergic to cinnamon.
. More than two glasses of orange juice in 24 hours makes them howl in harmony with alarms and sirens.
. Celery makes them sad.
====

Probably we should ask the Earmark team if they want to support this feature there, but I want to know your opinion before proceeding .

[josevalim]

HI @milmazz,

I believe this could be useful but there are some things to keep in mind. First of all, this is not supported by Markdown nor Commonmark. So it means people may try using this on their README and it won't work.

I personally find the Python-Markdown syntax quite foreign. I like the asciidoc syntax though. The best here would probably to come up with a syntax that would be nicely formatted on markdown when undetected but it could be detected and improved if desired.

For example:

[WARNING] This is a warning text.

And:

> ### [WARNING] Do not do this!
>
> This is the warning text.

So if Earmark sees a blockquote starting with a header followed by [ADMONITION], then it can render it in special content. As we can see, those are rendered just fine by Github:

[WARNING] This is a warning text.

And:

[WARNING] Do not do this!

This is the warning text.

[OvermindDL1]

<slightly-offtopic-but-related-to-admonitions>

Github handles restructuredtext fine, it's an old format and handles about everything in existence, including the above. I'd personally love to see ex_doc support ReST as well and things like this would come for free, as well as a log of other things, including (even optionally) parsed blocks of code for tests with even specific 'header' code that can be executed but not shown. It is the general 'Code Documentation Format', sphinx uses it as it's primary format as do many many other systems. All without needing custom markdown extensions.

[josevalim]

Unfortunately, having the community change as a whole to another format is way to complicated, so a markdown extension is the most accessible way to go. :)

[OvermindDL1]

Oh not change at all! I mean have ex_doc support multiple formats. :-)

There is precedence for this, 'most' programming language documentation systems support multiple languages, ReST tends to be the most common Default one in those, but multiples are supported. Even using an external tool like the ever-popular Pandoc you could support a few dozen formats and output them all to a single format that ex_doc would then handle exclusively (say, ReST due to it's abilities for code handling).

Could specify the document format on the @doc line somehow, perhaps like:

@doc format: :rest
@doc ""

# or perhaps

@doc format: ReST, do: """
"""

It could be started as a new alternative library to ex_doc, but that's starting to split abilities. What if ex_doc just had the ability to have pluggable input formats that output a native ex_doc format for output, kind of like how it has pluggable outputs? The above format: ... keyword argument could specify what format plugin to use for this @doc block with a default settable in the mix.exs or config.exs or so.

Even this Admonition support could just be tested as a new input support type.

me has a side thought of 'programming' exr files that are just ReST with code and test blocks that become the actual code and the rest is documentation, some other languages have such identical libraries, hmm

But really, ReST is so superior to markdown for a documentation format, tons of built-in features designed for documentation use and it is entirely pluggable with new features, all while still remaining readable as raw source like markdown.

Although admonition text block's could be added to the markdown processor for now, when does it stop with adding new features? Would you be amenable for ex_doc having pluggable input formats? The pluggable input format would even be useful for @tmbb and his project that keeps documentation 'outside' source files. ^.^

[josevalim]

The new documentation chunk does support custom formats (they are per module though) so this will eventually become supported but it is not our focus right now.

[tmbb]

ReST [...] is the general 'Code Documentation Format', sphinx uses it as it's primary format as do many many other systems.

ReST has its own limitation, which are not trivial (particularly regarding inline styles). In that aspect, I think asciidoc might have more features, although asciidoc's syntax is much more complex (IRC, ReST is context-sensitive but relatively simple).

Github handles restructuredtext fine, it's an old format and handles about everything in existence, including the above.

It is old, handles everything and has a total of one supporting implementation, which happens to require a VM which is not the BEAM. That's not a showstopper, but it's quite a big argument against.

But really, ReST is so superior to markdown for a documentation format, tons of built-in features designed for documentation use and it is entirely pluggable with new features, all while still remaining readable as raw source like markdown.

Indeed, but that comes at a cost of requiring something like 4 passes over the documentation. And requires python, whihc AFAIK is the only supported implementation.

What if ex_doc just had the ability to have pluggable input formats that output a native ex_doc format for output, kind of like how it has pluggable outputs?

It already does... Since my experiments with alternative syntax highlighters, ExDoc can support a custom markdown implementation. The thing is... It doesn't use the fact that it's a markdown implementation anywhere. It just assumes that's something that takes up text and outputs HTML. This means you can create your own "markdown implementation", which is actually a program that converts any format into HTML.

I almost suggested at the time that references to Markdown should be replaced in ExDoc in favor of something like "documentation format" or something like that, but then I had other stuff to do, and let it pass.

me has a side thought of 'programming' exr files that are just ReST with code and test blocks that become the actual code and the rest is documentation, some other languages have such identical libraries, hmm

You have my Guaxinim library (unfortunately not compatible with Elixir 1.7 because of some unannounced compatibility breaks which break the Amnesia package, one of which I can't quite pinpoint) which support inverse literate programming. You write normal Elixir files and the comments are converted into markdown prose.

[OvermindDL1]

I think asciidoc might have more features, although asciidoc's syntax is much more complex (IRC, ReST is context-sensitive but relatively simple).

Hmm, I use both and I'm unsure about that, ReST has been able to do things asciidoc hasn't, but I've not see anything asciidoc can do that ReST can't, though that is in my own experience.

And yep, ReST is context-sensitive, one reason why plugins are so much easier to write for it.

It is old, handles everything and has a total of one supporting implementation, which happens to require a VM which is not the BEAM. That's not a showstopper, but it's quite a big argument against.

What does? ReST has quite a few implementations, pandoc's for example is all built in haskell and compiled into pandoc itself.

Indeed, but that comes at a cost of requiring something like 4 passes over the documentation. And requires python, whihc AFAIK is the only supported implementation.

What about python? o.O?

It just assumes that's something that takes up text and outputs HTML.

Hence why I think it needs some intermediary format, something docbook-like but tuples/lists instead of xml perhaps.

This means you can create your own "markdown implementation", which is actually a program that converts any format into HTML.

What about converting to other formats, like epub, or pdf, or man pages, or docbook for integration into something larger, or to LaTeX, or to JSON for parsing in detail, all of which something like pandoc supports the output of.

Not saying to use Pandoc, but once you start adding non-standard extensions like admonitions into Markdown it becomes much harder to make use of such tools compared to using a language that already supports the features you need.

I wouldn't be opposed to asciidoc though, it's not bad, I use it as a templating language for the HTML on my maven servers for the users.

I almost suggested at the time that references to Markdown should be replaced in ExDoc in favor of something like "documentation format" or something like that, but then I had other stuff to do, and let it pass.

+1

You have my Guaxinim library (unfortunately not compatible with Elixir 1.7 because of some unannounced compatibility breaks which break the Amnesia package, one of which I can't quite pinpoint) which support inverse literate programming. You write normal Elixir files and the comments are converted into markdown prose.

Yeah your project is why that is on my mind again. ^.^

[josevalim]

Folks, the discussion has been completely side-tracked. Can we please move the discussion about other formats elsewhere? -- *José Valimwww.plataformatec.com.br <http://www.plataformatec.com.br/>Founder and Director of R&D*

[eksperimental]

It gives me the creeps to start adding features that are only going to be supported by only one parser.

I agree with @josevalim 's comment.

EDIT: But the idea of these text blocks is good. I like it.

[josevalim]

@RobertDober is this something you would be interested in adding to Earmark? Is it doable? Or should we try to implement it as a pass or using some other extension point? Thank you for your work!

[RobertDober]

A first thought that springs into mind is to just use an IAL

> [WARNING] Do not do this!
>
> This is the warning text.
{:.admonition}

however who is going to provide the stylesheet, ex_doc? (pushing the work back to you guys 😉 )

A second thought is to write a plugin, here is (the only) plugin, as an example.

https://www.hex.pm/packages/earmark_tag_cloud
If this does not cut it I'll be happy to consider alternative approaches, but I am against emitting css in Earmark.

[josevalim]

@RobertDober the CSS should totally be up to us, we just need a way to identify those with CSS rules. So if the blockquote has any kind of annotation, we can address it.

[RobertDober]

so either you use IAL, or we transform

> ### [WARNING] Do not do this!
>
> This is the warning text.

into

<blockquote class="admoniton"><p>WARNING...

of course the first p could get helpful attributes too.

I am quite sure that @pragdave would not object to implementing this, nor would YHS 😉

EDIT: I just do not know Markdown 🤣