Allow handing in pre-parsed AST

[pke]

For my component I'd like to extract all the text links from the markdown to build an action sheet out of the links.

So I would like to pre-parse the markdown using markdown-it for all links and then hand in that AST to the component instead of a markdown-it prop.

I want to prevent thet parsing of the markdown twice, once in my component and second in yours.

Do you think that would be a possible addition?

[iamacup]

So we could do two things I think

1. Make a prop that accepts the output of markdownit.parse - if this exists we just use this instead of running it internally
2. Make a prop that is the AST, which would replace the output of AstRenderer.js if present

The problem with 2 is, AstRenderer.js is a very changed file - so there is potential for lots of breaking in the future.

What do you think?

[pke]

The output of markdownit.parse is almost what I am using now, but I prefer a less nested AST (preferably flat even) so I am using your exported tokensToAST.

I am currently using this code and it works great but has a performance hit due to double parsing:

const links = useMemo(() => {
  const markdown = MarkdownIt({
    html: false,
    typographer: false,
  })
  const ast = tokensToAST(stringToTokens(children, markdown))
  const collectLinks = (result: FinePrintLink[], node: ASTNode): FinePrintLink[] => {
    if (node.type === "link") {
      result.push({
        uri: node.attributes.href,
        text: node.children?.map(({ content }) => content).join(" "),
      })
    }
    return node.children.reduce(collectLinks, result)
  }
  return ast.reduce(collectLinks, [])
}, [children])

I would not introduce a new prop, but rather define children additionally as union with ASTNode[] or even the return value of markdownit.parse

So my call to Markdown could look like this:

const ast = tokensToAST(stringToTokens(children, markdown))
<Markdown onLinkPress={onPress} style={style}>{ast}</Markdown>

Could also be possible

const tokens = stringToTokens(children, markdown)
// same as
const tokens = MarkdownIt().parse(children)
<Markdown onLinkPress={onPress} style={style}>{tokens}</Markdown>

What do you think?

[iamacup]

Hey

Sorry about the delay here - can you take a look at https://github.com/iamacup/react-native-markdown-display/releases/tag/6.2.x and let me know if this is what you were thinking?

Supports this syntax:

const markdownItInstance = MarkdownIt({typographer: true});

const copy = `
# H1

Paragraph
`;

const ast = tokensToAST(stringToTokens(copy, markdownItInstance))

            <Markdown
              debugPrintTree
            >
              {ast}
            </Markdown>

[iamacup]

released this in 6.1.5 - re-open if we need to change how it works :) Readme is updated to reflect the above

[pke]

Thanks, I'll test it on Monday and report back here!

[pke]

@iamacup works like a charm! Thanks for fulfilling this feature request!

[helsonxiao]

Two problems in 6.1.6:

1. Passing in AST nodes is slower than plain markdown string, it's obvious in FlatList scenario. (even the AST nodes is not modified).
2. Image node can't be parsed in AST nodes way.

p.s.
Currently I'm using React.memo for FlatList item cache, so problem 1 is not that serious, but it's really slow.

[pke]

@helsonxiao thanks for reporting back.

to 1st: Could you provide a source snippet how you use the AST within the FlatList? The markdown component also uses just memoization internally so maybe your AST is re-calculated over and over again because of unstable dependencies?

to 2nd: You mean images are not displayed that way? That shouldn't make any difference. Maybe a test case could be made for that.

[iamacup]

just opening to keep track of this in case it turns out to be library related - thanks for handling @pke !!

[helsonxiao]

@helsonxiao thanks for reporting back.

to 1st: Could you provide a source snippet how you use the AST within the FlatList? The markdown component also uses just memoization internally so maybe your AST is re-calculated over and over again because of unstable dependencies?

to 2nd: You mean images are not displayed that way? That shouldn't make any difference. Maybe a test case could be made for that.

1. I haven't measured it with Performance API but I do feel the difference. I also tried to cache the AST processor or return AST nodes directly. Note: My FlatList Item use react-native-syntax-highlighter for code display, but it's still slower than plain string mode after I removed the highlighter. If it's still confused for you, I will check your codes. Anyway, React.memo is necessary so it's not a big deal.

2. Try this input. It works well in plain string mode.

const input = "![背包四讲.png](https://media-test.jiuzhang.com/media/markdown/images/7/16/72644746-c710-11ea-8f89-0242ac1c0003.jpg)\n\n```\n[[java]]\nasfafagagag233\n[[golang]]\nasfgsagag\n```\n\n![背包四讲.png](https://media-test.jiuzhang.com/media/markdown/images/7/16/72644746-c710-11ea-8f89-0242ac1c0003.jpg)"

// turn on astPreprocessor then images disappear
return (
  <Markdown /* astPreprocessor={astPreprocessor} */ rules={rules}>
    {input}
  </Markdown>
);

// here I just return the ast, nothing changed.
// I will use this method to pick up important nodes.
const astPreprocessor = (ast: ASTNode[]) => {
  return ast;
};

const Markdown: React.FC<MarkdownProps> = props => {
  const _props = {
    rules,
    style: MarkdownStyles,
    ...props,
  };

  if (props.astPreprocessor) {
    const ast = tokensToAST(stringToTokens(props.children, markdownItInstance));

    return <_Markdown {..._props}>{props.astPreprocessor(ast)}</_Markdown>;
  }

  return <_Markdown {..._props}>{props.children}</_Markdown>;
};