Two new editor workloads

[bgrins]

This is coming out of the research into the rich text editing scenario and the meeting discussion last week. Here are two new workloads - one for code editing (CodeMirror) and one for WYSIWYG editing (TipTap).

They can be tested with the patch applied by loading http://localhost:7000/?suite=Editor-CodeMirror and http://localhost:7000/?suite=Editor-TipTap.

The basic structure here is a new subfolder which has very similar functionality across any supported editor (with the idea that we will plug one or two more in once we're happy with the overall approach). It's a simple Vite project with built artifacts committed in the dist subfolder. I built a small API for each editor (codemirror.js and tiptap.js) and to keep things simple have just duplicated most of the test page itself for each rather than trying to share a lot of code between them - codemirror.html and tiptap.html both have a <script type=module> which import the corresponding library API along with handlers for the buttons on the page which are used to drive the test itself from tests.mjs.

The test waits for initialization to complete, and then measures the time to load fairly large text content, then formatting it (syntax highlight or bolding), then scrolling to the bottom. There are a number of variables which could be tweaked - for example the size/complexity of the text content, whether we measure "unformatting" or changing to smaller text, and whether we force a layout as part of each individual step. Happy to discuss any/all of them here or in follow ups.

Finally, what about Monaco? We've agreed to add Monaco (in i.e. #5) but unfortunately I've struggled to make a deterministic test case so far. I believe there is a missing "async step" capability in the current runner that will block integration here. The reason is that it uses workers for the editor and for language highlighting. I was running into problems where one or both of those workers were causing network requests in the middle of tests (obviously bad, but something that could likely be solved with a better understanding of the API). When I simulated fixing that with i.e. a timeout, then I saw other problems like a random amount of worker effort being included in the measured time (which presumably penalizes browsers which get more processing done in the language worker as that would translate into main thread highlighting). Ultimately I felt this was a separate effort to deep dive on the runner and the library, so didn't want to block landing a couple other workloads in the meantime. I did adapt the shared API to expect async initialization since this will ultimately be required for Monaco and potentially other editors.

[bgrins]

@rniwa for some reason this particular workload seems slow on WebKit, at least on my laptop. I don't think it's specific to how it's integrated within the harness (i.e. I also see jank loading https://tiptap.dev/examples/book), but if you see anything that looks wrong with the test itself let's discuss. Note the PR is only using the first chapter and not the entire book - I haven't tested what happens with much longer text.

[rniwa]

Yeah, I looked into it but I don't see anything obviously wrong.

[flashdesignory]

This looks really clean ✨ . I like that both editors are using the same api to interact with, which makes it easy to compare.

[bgrins]

@rniwa do you want to have a look now, or should I land it in tentative?

[rniwa]

@rniwa do you want to have a look now, or should I land it in tentative?

Let me look into it today.

[bgrins]

Another thing that would be quite nice to capture if possible is typing additional characters, to make sure we cover more incremental modifications as well as the big setValue call.

The Monaco GrandPrix test did this by setting the value on the hidden textarea and then simulating some DOM events for the framework to pick it up. From quick testing it seems like something like this works for these contentEditable based editors: page._frame.contentDocument.execCommand('insertText', false ,'// text;\n');.

[rniwa]

So scrolling steps seem to always measure 0ms in all browsers. Maybe it's not testing what we want to test since scrolling happens in non-main thread? I suggest we get rid of those steps.

[rniwa]

Having said that, it seems okay to land this to tentative directory.

[bgrins]

So scrolling steps seem to always measure 0ms in all browsers. Maybe it's not testing what we want to test since scrolling happens in non-main thread? I suggest we get rid of those steps.

You're right. I don't think this was the case previously - the frameworks tend to detect the scroll and do some virtualized rendering to update the DOM, and I believe it was catching that in the async timing in the past. So either I was mistaken about this, or something changed (possibly in the step ordering or size of the text).

[bgrins]

Will get back to this next week. Want to make sure things are working as expected with the scroll step removed.

[bgrins]

It took me a while to debug but I was seeing something weird in Firefox with network requests happening inside the timed region. Turns out this was due to the module preload polyfill in Vite (https://vitejs.dev/config/build-options.html#build-modulepreload). This likely affects Safari as well.

We clearly don't need preloading here, since it's only trying to optimize static imports in the module (which will be loaded and resolved before the test starts), so I've disabled that in the build.

[bgrins]

Alright, pushed up various small updates after manually auditing. The one substantive change I think we should make before pushing to tentative and allowing everyone to take a deeper look in the tree is including the editor initialization inside of the measured time as a step.

Because Monaco wasn't well suited for this, I ended up making the creation async and happen entirely in the prepare phase. These two editors in particular do not have async creation though, and it seems most realistic to capture the actual work required to instantiate the editors since that's part of the work required in order for them to become interactive for users on real pages.