Task Scheduling

[slightlyoff]

The CSS Houdini effort is looking to specify a pipeline model for execution of script in layout/raster: https://docs.google.com/document/d/1Mw6qNw8UAEfW96CXaXRVYPPZjqQS3YdK7v57wFttAhs/edit?pli=1#

This (re)raises the question about ordering for delivery of other tasks in the platform (Mutation Observer change records, Object.observe records, setTimeout callbacks, setIdleCallback callbacks, requestAnimationFrame callback delivery). This has been discussed at some length, e.g., with @wycats at TC39.

The open question here is if the TAG should try to untangle this and generate a recommendation for scheduling in the non-layout/raster bits of frame generation.

/cc @annevk

[annevk]

As @domenic said, CSS should probably refactor https://html.spec.whatwg.org/#processing-model-8 to make things work for them. Review would be good, but I suspect for quite a bit of it we're bound by compatibility.

[travisleithead]

Probably need to enumerate all the timings in the system, then figure out what a "tick" might be, and describe what feature areas understand which ticks, which APIs slot into which ticks. Whether there might be paired responses (dependency control). Task re-ordering? Lots of things to consider.

Should also look at various frameworks (like fastdom) and understand what scheduling policies are in use.

[torgo]

Taken up at Stockholm f2f.

[dbaron]

What we (mostly Travis) wrote on the whiteboard during the meeting:

[torgo]

Scheduled for 5/16 to work through it. We agreed to work on it in the London F2F but that we would need to do some prep work ahead of time. /cc @travisleithead

[torgo]

Discussed at London F2F - possible we might create a note that "explains how this all works together."

[travisleithead]

Some thoughts during discussion in the breakout session:

Potential problem: no opportunity to process input during the rendering loop (which some consider higher-priority than rendering)?
• Some implementations (i.e., Chrome) were considering processing input at the same rate as the Render-loop, but coalescing multiple input events of the same type (e.g., mouse/touch moves) into a single event (either dropping the others in the queue, or providing the additional data in the single event).

Invariants? What are they?

Background processing: (for things in background tabs, (or potentially in non-visible iframes, etc.) slowing the render-loop frequency (or enabling it to be time-sliced to process foreground work, can lead to increased queue sizes in the background tab, which can lead to 'jank' in the foreground tab when the background tab finally drains all the queues.

Where should certain processing be done relative to where layout is computed? Should be based on how the API might use layout info (e.g., wse layout data vs. set style data):
• RequestAnimationFrame - Seems like purpose is to write styles (vs. reading layout), thus should ideally be done before computed layout (or you would run layout twice if layout was read in the callback)
• IntersectionObserver - Purpose is to read layout, thus should be in the pipeline after layout has been done, if styles are written (then layout would be done twice).
• ResizeObserver - Hmm. Writes styles and reads layout--where to put it?

[travisleithead]

We would want some guideance in our design-principles doc, about where a callback should slot into the Rendering Loop, or whether it should be a new task. See w3ctag/design-principles#38

[travisleithead]

Pictorial view of HTML's render loop, with annotations for Edge noted in grey:

[dbaron]

So a few further thoughts here:

• anything that has callouts to script from within the rendering pipeline leads to the risk that we might have to repeat some of the core steps (e.g., style, layout) of the rendering pipeline within a single vsync / refresh tick
• we tend to want to put the things whose purpose is primarily to write (e.g., requestAnimationFrame) early in the process, and things whose purpose is primarily to read (e.g., IntersectionObserver) late in the pipeline. When they're used the other way around, they can of course end up triggering repetition of steps.
• When we end up doing things that require repeating some of the steps, do we run these things more than once? In most cases, probably not; we don't run the things that call out to script when there are layout or style flushes. However, in cases where the later steps whose primary purpose was expected to be reading end up instead making writes, we've now run other read steps with data that's now bad. Do we run them again? What if they keep looping around? (And how do they interact if there's more than one type of callout to script that we run again -- e.g., media query listeners on an iframe that poke style in their parent document and change the iframe's size, and say, an IntersectionObserver that does something similar?) I wouldn't be surprised if there's some pretty poor interop here.

Another way of thinking about the input processing issue is this: we want the rendering loop itself to be as small as possible so that we can get back to the event loop to do other things (e.g., processing input). It's also nice to tie some of the input processing to the refresh loop so that we can batch it and avoid unnecessary processing (e.g., mouse movement or touch coalescing). These two things seem to conflict a little bit, though maybe it's not a big deal. (Is it well-defined when microtasks and nanotasks run in response to any event dispatch that's tied to the rendering loop, or to any other callouts to user script?)

But, really, this issue is waiting for somebody to do the work of understanding and testing this.

(Sorry, this comment might not be completely intelligible, but we need to move on to a scheduled discussion in the meeting now...)

[domenic]

I just want to reemphasize that while it is good people are working to understand this section of the spec and implementations, I do not believe it's something that should end up in this design principles document. There are no design principles here to be had. Perhaps a blog post explaining things, or a pull request to the spec adding more non-normative explanatory text, or more tests to be written. But those are different.

[dbaron]

This issue isn't in the design-principles repo. It's the design-reviews (formerly spec-reviews) repo.

[domenic]

I guess I was over-extrapolating from

We would want some guideance in our design-principles doc, about where a callback should slot into the Rendering Loop, or whether it should be a new task. See w3ctag/design-principles#38

[dbaron]

This is related to #489... which I was actually briefly confusing with this issue.

[dbaron]

So @atanassov and I looked into this in a breakout at our virtual "Cork" face-to-face meeting.

There's clearly still stuff to improve about the platform here. But we've had this issue open for five years, and it hasn't caused a lot of progress to happen (I did write one test, at least). So we think it probably makes sense to close this not because it's fixed but because it's unlikely to lead to further progress. (That said; there are some other issues that may lead to some progress; see previous comment.) And it's still possible that I or someone else will find time to look into some of the remaining issues.