Supporting pandoc 2.11

[ickc]

There's a change in pandoc AST since 2.10. c.f. jgm/pandoc#1024, jgm/pandoc#6277 and https://github.com/jgm/pandoc/releases/tag/2.10

The new Underline is probably easy. The change of AST regarding table is huge (jgm/pandoc#1024) including support for span.

Edit: Links on new table AST

• PR
  • Feature request started here: Support for table column spans, table attributes in AST jgm/pandoc#1024, Improving tables jgm/pandoc-types#65
  • this is where the actual change of the AST happened: https://github.com/jgm/pandoc-types/pull/66/files?file-filters%5B%5D=.hs#diff-01f4ffe52cf097ab9ff89eebce394c86
  • pandoc 2.10 release note: https://github.com/jgm/pandoc/releases/tag/2.10
  • pandoc 2.11 release note (see HTML writer): https://github.com/jgm/pandoc/releases/tag/2.11
  • pandoc-types 1.21 changelog: https://github.com/jgm/pandoc-types/blob/master/changelog
• def
  • https://hackage.haskell.org/package/pandoc-types-1.22/docs/Text-Pandoc-Definition.html
  • doc on lua filters has some info on this: https://pandoc.org/lua-filters.html
  • See comments at https://github.com/jgm/pandoc-types/pull/66#issuecomment-611053332 and after on normalizing tables
• example ASTs:
  • https://github.com/despresc/pandoc-types/blob/master/test/test-pandoc-types.hs
  • https://github.com/jgm/pandoc/tree/master/test/tables

[sergiocorreia]

Thanks! This is indeed an important change.

First of all, I feel like I should just add support for id/attrib/classes to all elements, because we seem to be converging towards that.

For the tables, is there an actual summary of the changes of the new AST? That issue thread looks huge

[ickc]

More links related to the AST:

• Support for table column spans, table attributes in AST jgm/pandoc#1024
• Improving tables jgm/pandoc-types#65
• Better tables jgm/pandoc-types#66 : this is where the actual change of the AST happened
• https://hackage.haskell.org/package/pandoc

[ickc]

I feel like I should just add support for id/attrib/classes to all elements, because we seem to be converging towards that.

IIRC @jgm is reluctant to that idea. I don't think it will really happen, although many might ask for it. The solution seems to be just wrap whichever element one want to have attributes by a div and do it there.

[jgm]

If I were designing afresh, I'd have attributes on everything. But it would be quite a big change now.

[sergiocorreia]

@ickc by any chance do you have on hand any input file I can use to test the recent changes? I was initially thinking about having a markdown file with a complex table, but I understand that although the AST changed, the readers/writers haven't (yet).

So, are you aware of a way to receive and feed data into pandoc? One option is to try using a filter and seeing if pandoc accepts it, but seems a bit fragile.

[ickc]

I think may be the best way to obtain a test file would be starting from https://github.com/despresc/pandoc-types/blob/09cb4314010365abc4512c2363b83711c92ac18b/test/test-pandoc-types.hs. Here's the diff leading to this: https://github.com/jgm/pandoc-types/pull/66/files?file-filters%5B%5D=.hs#diff-01f4ffe52cf097ab9ff89eebce394c86. There's a JSON AST there that might be useful.

Other than the JSON, pandoc native AST could be a start. For short tables it is still manageable by hand.

[ickc]

Add some comments related to chdemko/pandoc-numbering#18 (comment)

According to https://github.com/jgm/pandoc-types/pull/66/files#diff-8c56b05df299ade82ea6265ee0b71bd0, these are added:

, Caption(..)
                               , ShortCaption
                               , RowHeadColumns(..)
                               , Alignment(..)
                               , ColWidth(..)
                               , ColSpec
                               , Row(..)
                               , TableHead(..)
                               , TableBody(..)
                               , TableFoot(..)
                               , Cell(..)
                               , RowSpan(..)
                               , ColSpan(..)

and TableCell is removed.

Then later,

		 -- | A short caption, for use in, for instance, lists of figures.
 type ShortCaption = [Inline]

  -- | The caption of a table, with an optional short caption.
 data Caption = Caption (Maybe ShortCaption) [Block]

So the ShortCaption is an Inline and Caption is block element.

[dilawar]

Is there a beta version for this feature? I have pandoc-2.10 installed and if possible if would rather not downgrade.

[NMarkgraf]

As I can not see any commits toward a solution to this problem, is there any way we could help?

[aubertc]

The reasons why this is more challenging than thought have been sketched by @sergiocorreia , one of panflute's author, at chdemko/pandoc-numbering#18 (comment) . If you find a good solution to the absence of test, I believe this could be extremely useful. You can also, probably, "hack" something, but there will be no guarantee that it will work on the long run (that is, when writers / readers will change too).

[NMarkgraf]

Okay, but so panflute is currently unusable ... I find that is also not a good idea...