Description
There is some concern over whether the new docstring code snippet formatting could slow down the overall formatting process. To ameliorate this and understand its performance characteristics, we should try to write some micro-benchmarks that will help us track performance over time. @MichaReiser in particular had some suggestions here: #8857 (comment)
I think we should try to include some "real world" examples of code snippets, but it might also be useful to try to include some "tortured" examples too. That is, try to devise the absolute worst case for performance and see how it does. This will give us an idea of what perf might look like at the extremes. For example, this might help us characterize the costs of us checking whether the reformatted code is valid or not. See #8857.
At present, I have done some very loose ad hoc benchmarking to get a general sense of things:
For an ad hoc benchmark, if I run the formatter with and without
docstring-codeenabled, then runtime is about the same. I ran it a few times and couldn't notice a difference. (Not that this is a good substitute for a real benchmark, but perhaps suggestive that docstring code formatting doesn't have a huge impact on perf.)
In particular, I did that on CPython and polars.