[TASK][MEDIUM] Unlock advanced features of markdown in docs

[pan3793]

What's the level of this task?

MEDIUM

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before creating

• I have searched in the task list and found no similar tasks.

Mentor

• I have sufficient expertise on this task, and I volunteer to be a mentor of this task to guide contributors through the task.

Skill requirements

• Sphinx, and maybe some HTML and CSS skills

Background and Goals

Kyuubi uses Sphinx to build the docs, and RST and Markdown are mixed-used.

RST is a much more powerful format but easy to mess up and hard to learn, while Markdown is more popular and easy to learn.

Currently, Kyuubi uses a legacy markdown==3.4.1 for Sphinx markdown support, which lacks some advanced features, e.g. variable substitution, which is likely to be used for "version"

While the recommended powerful markdown solution is myst_parser https://www.sphinx-doc.org/en/master/usage/markdown.html, we should investigate and migrate to this one.

Implementation steps

• configure Sphinx to use myst_parser
• make sure everything in docs looks well, especially the configuration page https://kyuubi.readthedocs.io/en/master/configuration/settings.html
• converting at least one RST docs to markdown which has variable substitution

Additional context

Introduction of 2024H1 Kyuubi Code Contribution Program

[PorterZhang2021]

Hello, Mr. Pan, I would like to know if anyone has worked on this issue? I am not very familiar with Kyuubi's documentation construction and Sphinx, so I may need to spend some time learning and researching this part. I plan to spend about an hour each day and report the progress once a week. Do you think this is appropriate? If it is suitable, you can arrange it for me. Thank you.

[pan3793]

@PorterZhang2021 go head, please.

[PorterZhang2021]

Ok，I will do it.If I have any new questions or come to the exhibition, I will talk to you on this issue. Thanks.

[PorterZhang2021]

Hello Mr.Pan @pan3793 , I tried using myst_parser for parsing and checked the Configuration page and found no issues. Overall, there were no problems. I also fixed some minor issues midway. What more work do I need to do in this area?