feat(texteditor): Markdown dual conversion for backward compatibility

[habibayman]

Summary

This PR includes the conversion of the structured JSON format TipTap uses to&from Markdown to maintain backward compatibility with the old editor's API.

supports the conversion for:

• Standard Markdown elements previously handled by the ToastUI editor and its Showdown converter.
• A specific, legacy format for custom nodes, particularly for Images (![alt](placeholder/checksum.ext =WxH)) and Math Formulas $$latex$$

The formats for the custom nodes are adapted from the old editor's standard syntax conversion.

I added unit tests to stress test both ways of the conversion .

Note

the new editor to use data URLs for image uploading here, I made sure the new editor can load previously uploaded server side images from the old editor too.
my concern is that data URLs are quite larger: (take up almost 30% more than the actual image size)
So, another PR will be created to use a server image uploading method.
UPDATE: done in #5271

As for right now, everything is working fine:

• Legacy images load & saved ✅
• New images load& saved ✅

---

References

This PR adds no changes to the UI that will remain permanently, just a playground to test in!

• part of the tracking issue: Update rich text editor to flexible, extensible rich text editing framework #5049
• fixes: [New Rich Text Editor]: Maintain backward Markdown Compatibility #5185

---

Reviewer guidance (Important)

• how to test loading markdown? (from Markdown)
  You can edit the SAMPLE_MARKDOWN variable in the fake parent and refresh the page to see it render well in the editor

• how to test updating markdown? (to Markdwon)
  You can straight up start editing in the editor and you'll be able to see the equivalent markdown in the Live Markdown Output (v-model state) container below.

• how to test with legacy content?
  Well, this isn't the best approach since the editor is not yet ready to fully replace the old editor (very soon though — just not in this PR!), but it is good enough for the specific testing purposes of this PR.

1. I created an exercise in a channel
2. created a question with its answers
3. replaced the with and watched the answers I previously wrote using the old editor displayed formatted in the new editor.

[nucleogenesis]

I've tested things out locally w/ images and various markdown updates. I did some testing around how things work when there is a lot of text.

My not very rigorous test suggests that things work well up to a pretty significant document size - but once I'd copy-pasted the default text in there a few dozen times it started to get a bit laggy - which I think is to be expected at that point 😅

I added a couple of notes on code review and will give a little more time to the markdown serialization, but all-in-all this feels and looks great!

[nucleogenesis]

What is the use case for the parent component changing the content? Could/would this happen while I'm actively editing the content?

[habibayman]

Currently no, it's not used anywhere.
I wrote it because my goal was to mimic a similar communication way to the one the old editor had with its parent.
The old editor didn't use V-bind but the data inside it was passed in both:

• Down(Parent to Editor)
  The editor listens for changes on the markdown from the parent.
  Check this piece of code.

• Up(Editor to parent)
  Native library event (editor.on('change', ...)) -> Vue event (this.$emit('change', ...)) -> Parent handler (@change="...")

I recreated the same logic using a different pattern,
I believe the "communication" method part finalization belongs more to the editor replacement issue #5206
so we'll be talking more about this in that PR giving it a closer look.

[nucleogenesis]

Curious for why this is being delayed 500ms before being called here. Do we need there to be at least 500ms between the function call and it's execution - or just for it not to be called more than once every 500ms?

If it'd be better for this to be evaluated immediately, but only once every 500ms, I suggest using _.debounce here. This way it'll execute immediately, but only if it's been >=500ms since the last time it was called. It'll also remove the need to manage the lifecycle of the timeout objects.

[habibayman]

not to be called more than once every 500ms

Yes.
I do this so I don't clatter the undo/redo stack. The saveSize() function causes an editor state update which by default gets added to a stack.
If I keep spamming it with updates, using the undo button after that will be a nightmare!

Thank you for the function suggestion, this is the first time I hear about it :) refactored in 48a0bc4

[habibayman]

Can you run the linter and see if you can replicate having similar lines in the console?

Error while parsing /home/habiba/Downloads/studio/node_modules/marked/lib/marked.esm.js
Line 247, column 12: Unexpected token ;
`parseForESLint` from parser `/home/habiba/Downloads/studio/node_modules/.pnpm/vue-eslint-parser@9.4.3_eslint@8.57.1/node_modules/vue-eslint-parser/index.js` is invalid and will just be ignored

My first guess was that this line must be the problem since this branch is still on node 18.20.7 ; so I tried downgrading marked to a version that uses node >= 18 which should work, but that changed nothing.

[habibayman]

I think this will be fixed when merged into unstable that uses a newer version of node 👍🏻

[habibayman]

@nucleogenesis

which I think is to be expected at that point 😅

This may be the cost of all these regular expressions? I think it's tolerable though.

I added unit tests for loading/saving markdown, and added a test for a large input to see whether it will tolerate that or not, ($\color{rgba(0, 256, 0)}{\textsf{ PASS }}$) 👍🏻

[nucleogenesis]

@habibayman Overall this is looking great - the additional tests are solid and cover the serialization well, thanks for adding those.

I think that this is good to go - I'll hold off on the final Approval until great mergening is upon us.

[marcellamaki]

Officially approving now after merging to studio has been unblocked