Apply `ruff` to `markdown` code blocks

[paddyroddy]

It would be cool to have the functionality to run ruff over python codeblocks like this does for black https://github.com/adamchainz/blacken-docs

[JonathanPlasse]

https://github.com/pydantic/pytest-examples
Does something similar.

[evanrittenhouse]

I'm interested in trying to implement this. Looking at the projects listed above, it seems like code blocks are determined via Regex. Do we want to copy that approach or use a more robust Markdown parser (pulldown-cmark seems like a popular one)?

[JonathanPlasse]

I would vote to use a markdown parser for robustness as you suggested.

[MichaReiser]

Using a markdown parser makes sense from my view. It may be worth to also consider tree-sitter because it supports many languages.

The part that's unclear to me how we want to solve it in the short term is the mapping of column and line numbers in the Messageemitters. Ruff supports Jupyter-notebooks today, but the line-mapping is somewhat "hacked-in" to make it work, and I'm a bit reluctant to add more exceptions to the line-number mapping.

https://github.com/charliermarsh/ruff/blob/483f4799995d1aea7987e6f7ee403f0411060023/crates/ruff/src/message/text.rs#L70-L87

I assume a similar mapping will be necessary for Markdown files because we only pass the code block's source to Ruff, but we should show users the absolute line number from the start of the Markdown document.

Long-term

Long-term, we'll need to support the following two features.

Multi-language support

Ruff should support linting different file types. For example, Ruff should be able to lint SQL, python, and Markdown files. The filetype decides which specific linter Ruff uses to lint the file. This includes that the LSP is in sync with the file-types supported by the CLI and what extensions map to which languages. Rome supports this today.

Embedded Languages

The idea is that a document written in one language can contain code written in another language. Examples are:

• Code blocks in markdown documents
• SQL in Python code
• Python code in Jupyter Notebooks

The markdown file handler would recognize code blocks and delegate to Ruff to decide how to parse, lint, and format the code block's content. Ruff's infrastructure would correctly map the line-numbers between the "virtual" documents and the "physical" documents on disk.

[evanrittenhouse]

Yup, agree with all of your points @MichaReiser. As part of the design phase I've been thinking about how to generalize to different filetypes. It's an interesting problem - I will take a look at Treesitter parsers as well. Going down that route, we could support different languages relatively easily. I am going to submit PRs for this incrementally so that we can iterate over the design. The first one will be a parser that pulls code blocks out of Markdown - I'm leaning towards using Treesitter since that architecture could make it easier to provide "plugins" for other languages in the future.

As far as the line mappings, I was planning on making a generic LineMap struct (or something along those lines) which would contain line numbers mapped to their offset in the containing document. That struct could then be reused for embedded languages, like you mentioned, mapping relative line numbers in the chunk of interest to the absolute line number of the document. It should provide a flexible way to treat code blocks differently, regardless of if it's an embedded language, a markdown code block, a Jupyter cell, etc.

I'll have to look into Messages and how they work before finalizing the code though. I've been reading through the Jupyter implementation as part of this.

[MichaReiser]

As far as the line mappings, I was planning on making a generic LineMap struct (or something along those lines) which would contain line numbers mapped to their offset in the containing document.

Mapping line numbers should suffice for markdown documents but wouldn't be sufficient for e.g. SQL in python. I think it may actually be sufficient to simply add the byte offset of the markdown block to the range of every message/diagnostic.

[evanrittenhouse]

Yup, that's basically what I'm doing - adding a document_offset field to Messages and Diagnostics.

As far as the SQL in Python example, I was thinking the approach could work because the document_offset would be where the SQL code starts relative to the Python file. Line 1 of the SQL code could be on line 50 of the overarching .py file, so the document_offset for all diagnostics/messages in that block would be 50. Do you think there are flaws in that approach?

E: sorry, not mapping line numbers. Mapping byte offsets between SQL/Python lines. I still need to figure out a concrete design though, if you couldn't tell 😅

[evanrittenhouse]

Going with a treesitter integration requires installing the treesitter grammar for the language in question. While we could gate this behind a workspace feature flag, I'd like some other folk's thoughts on it. I don't want to needlessly increase the binary size. I need to figure out how it all works to give exact an estimate of the increase, though.

Advantage of pulldown-cmark as mentioned above is it's quite fast and won't affect the binary size. Disadvantage is that it obviously can't be used for anything except Markdown.

@MichaReiser @JonathanPlasse do you guys have any thoughts?

[JonathanPlasse]

It seems easier to first handle only markdown and in a second time expand with the tree-sitter grammar.

[MichaReiser]

Yup, that's basically what I'm doing - adding a document_offset field to Messages and Diagnostics.

Which use cases require a diagnostic or message to know its document offset? Would it be sufficient to mutate the diagnostic.range or message.range directly by adding the offset? I'm asking because increasing the size of Message and Diagnostic decrease performance because:

• More data that must be written to and read from the cache
• Overall more data to write when creating/copying diagnostics and read when generating the output (even for diagnostics not using embedded languages)

Advantage of pulldown-cmark as mentioned above is it's quite fast and won't affect the binary size.

My expectation is that pulling in pulldown-cmark increases the binary size because we include a new crate in our binary (that is 116 kB in size), or is pulldown-cmark already a (runtime) dependency ?

Going with a treesitter integration requires installing the treesitter grammar for the language in question.

Yeah, using treesitter is more complicated because it requires a custom build step. I'm not too opinionated on if we should use treesitter or not. I think we can start prototyping with either and defer the decision to later.

[evanrittenhouse]

Would it be sufficient to mutate the diagnostic.range or message.range directly by adding the offset?

Yep! I was working on it yesterday and that's also the route I've chosen to go down. Sorry for the confusion, I should've waiting until I had a fully fleshed out design before weighing in.

My expectation is that pulling in pulldown-cmark increases the binary size because we include a new crate in our binary (that is 116 kB in size), or is pulldown-cmark already a (runtime) dependency ?

It's not a runtime dependency, so it'll increase the crate because of its size, yes. I meant that it'd be less than the size delta from installing treesitter and associated grammar(s) which aren't totally necessary at this stage.

I think we can start prototyping with either and defer the decision to later.

Cool!