Atomic move operation for element reparenting & reordering

[domfarolino]

What problem are you trying to solve?

Chrome (@domfarolino, @noamr, @mfreed7) is interested in pursuing the addition of an atomic move primitive in the DOM Standard. This would allow an element to be re-parented or re-ordered without today's side effects of first being removed and then inserted.

Here are all of the prior issues/PRs I could find related to this problem space:

• Reparenting iframes without reloading html#5484
• Side effects due to tree insertion or removal (script, iframe) #808
• Strange behavior with insertBefore vs appendChild and transitions #880
• Proposal: New method to reorder child nodes #891
• Add before removing steps #1185
• Use a queue of deferred steps in the node insertion algorithm #732
• Use the new definition of the insertion steps html#4354
• DOM: Add iframe insertion & removal steps WPTs web-platform-tests/wpt#44308
• Add insertion tests for the new changes with the "deferred steps queue" web-platform-tests/wpt#15264

Problem

Without an atomic move operation, re-parenting or re-ordering elements involves first removing them and then re-inserting them. With the DOM Standard's current removal/insertion model, this resets lots of state on various elements, including iframe document state, selection/focus on <input>s, and more. See @josepharhar's reparenting demo for a more exhaustive list of state that gets reset.

This causes lots of developer pain, as recently voiced on X by frameworks like HTMX, and other companies such as Wix, Microsoft, and internally at Google.

This state-resetting is in part caused by the DOM Standard's current insertion & removal model. While well-defined, its model of insertion and removal steps has two issues, both captured by #808:

1. Undesirable model: The current DOM Standard allows for the non-atomic insertion of multiple nodes at a time. In practice, this means when appending e.g., a DocumentFragment, script can run in between each individual child insertion, thus observing DOM state before the entire fragment insertion is complete.
2. Interop issues: While Safari matches the spec, Chromium & Gecko have a model that ensures all DOM mutations are synchronously performed before any script runs as a result of the mutations.

What solutions exist today?

One very limited partial solution that does not actually involve any DOM tree manipulation, is this shadow DOM example that @emilio had posted a while back: whatwg/html#5484 (comment) (see my brief recreation of it below).

But as mentioned, this does not seem to perform any real DOM mutations; rather, the slot mutation seems to just visually compose the element in the right place. Throughout this example, the iframe's actual parent does not change.

---

Otherwise, we know there is some historical precedent for trying to solve this problem with WebKit's since-rolled-back "magic iframes". See whatwg/html#5484 (comment) and https://bugs.webkit.org/show_bug.cgi?id=13574#c12. We believe that the concerns from that old approach can be ameliorated by:

• Fixing Side effects due to tree insertion or removal (script, iframe) #808, as to not leave elements in a disconnected state at all during the atomic portion of the move (i.e., the re-parenting/re-ordering)
• Not allowing atomic moves across documents, which should greatly simplify the security story of this work

How would you solve it?

Solution

To lay the groundwork for an atomic move primitive in the DOM Standard, we plan on resolving #808 by introducing a model desired by @annevk, @domfarolino, @noamr, and @mfreed7, that resembles Gecko & Chromium's model of handling all script-executing insertion/removal side-effects after all DOM mutations are done, for any given insertion.

With this in place, we believe it will be much easier to separate out the cases where we can simply skip the invocation of insertion/removal side-effects for nodes that are atomically moved in the DOM. This will make us, and implementers, confident that there won't be any way to observe an inconsistent DOM state while atomically moving an element, or experience other nasty unknown side-effects.

The API shape for this new primitive is an open question. Below are a few ideas:

• A new DOM API like replaceChildAtomic()/replaceChildrenAtomic() that can take a connected node and atomically re-parent it without removal/insertion side-effects.
  • One limitation here is that we'd have to pick and choose which existing DOM APIs we want to mirror with atomic counterparts. For example, if we ever wanted append() or appendChild() to ever be able to also atomically move already-connected nodes, we'd have to introduce appendAtomic() and appendChildAtomic(), and so on.
• A setting for existing DOM APIs, e.g., append(node, {atomic: true}), replaceChild(node, {atomic: true})
• A scoped, declarative attribute that changes the behavior of DOM mutation APIs in a subtree
  • This could be an element attribute that makes all existing DOM mutation APIs behave "atomically" when operating on already-connected nodes under the element's subtree
  • This could also be a property on the document overall, set via a header/meta tag, or some other mechanism

---

Compatibility issues here take the form relying on insertion/removal side-effects which no longer happen during an atomic move. They vary depending on the shape of our final design.

1. With a new DOM API/setting that developers have to affirmatively opt-into, you could atomically move fragments/subtrees constructed by other library code that's unaware it's being atomically moved. Those fragments may be built in a way that relies on non-atomic move side-effects (though we haven't heard of such concerns directly yet).
2. Consider an element attribute that changes the behavior of all DOM mutation APIs to behave atomically on already-connected nodes in its subtree. You could minimize compat concerns by externally-constructed portions of the subtree to opt-out of atomic moves with the same attribute. But what would that mean exactly, to have part of a subtree move atomically and part of it not?

A non-exhaustive list of additional complexities that would be nice to track/discuss before a formal design:

• How to handle mutation events? There was discussion at the TPAC 2023 about suppressing mutation events when new-ish DOM features are used, so we could probably get away with simply suppressing mutation events whenever an atomic move is being performed??
• Handling things like focus/selection properly (need to land on desired behavior)
• Fixing up things like live ranges; the way DOM handles this today might already be suitable for atomic moves, but unclear

Anything else?

No response

[WebReflection]

First of all, thank you! I've been vocal about this issue about forever and part of one of the biggest discussions you've linked.

As author of various "reactive" libraries and somehow veteran of the "DOM diffing field", I'd like to add an idea:

The API shape for this new primitive is an open question. Below are a few ideas:

I understand a node can be moved from <main> to an <aside> element and this proposal should still work but I think we should not discard the Range API:

• most modern libraries have a concept of fragments, inevitably represented as virtual because there's no persistent fragment whatsoever yet on the DOM (I've been vocal about this too)
• in a classic table sort mechanism there could be only few TRs moved within a specific place and taht's the same for LIs and others ... if any proposed API consider only parentNode to work that would not satisfy most fragment based requirements where areas are confined within Virtual DOM or comment nodes to confine those special cases while the Range api could instead simply select a node start, a node end, and update atomically inner nodes

On top of this I hope whatever solution comes to mind works well with DOM diffing, so that new nodes can even pass through the usual DOM dance when the parent is changed or they become live, removed nodes that won't land anywhere else would eventually invoke disconnectedCallback if Custom Elements, but nodes already present in that container and moved around basically do nothing in terms of state, they are just shuffled in the layout, if they do.

As quick idea to eventually signal a node is going to be moved in an atomic way, and assuming it's targeting also a live parent, I think something like parent.insertBeforeAtomic(node[, reference]) could be an interesting approach to consider as that basically solves everything, from append to prepend to any other case insertBefore works wonderfully well and it hints that such node should:

• do nothing if the parent is the same as before (or the node was already live) ... just move it and skip all the things
• trigger connectedCallback if the node was not live
• ... that's it?

As insertBefore covers append, appendChild, prepend, before and after with ease, it might be the easiest starting point to have something working and useful for the variety of virtual fragments based solutions and diffing APIs out there.

I hope this answer of mine makes sense and maybe trigger some even better idea / API.

edit on after thoughts another companion of the API should be reflected in MutationObserver, or better, MutationRecord ... so far we have addedNodes and removedNodes but nothing about movedNodes which will still be desired for most convoluted edge cases.

The movedNodes record will contain, beside of course the target, a from parent container and a to parent container which might be the same if moved internally but it would signal previous parent and new parent otherwise that something different is within their content.

[1cg]

This would be a fantastic addition of functionality for web development in general and for web libraries in particular. Currently if developers want to preserve the state of a node when updating the DOM they need to be extremely careful not to remove that node from the DOM.

Morphing (https://github.com/patrick-steele-idem/morphdom) is an idea that has developed around addressing this. I have created an extension to the original morphdom algorithm called idiomorph (https://github.com/bigskysoftware/idiomorph/) and the demo for idiomorph shows how it preserves a video in a situation when morphdom cannot. 37Signals has recently integrated idiomorph into Turbo 8 & Rails (https://radanskoric.com/articles/turbo-morphing-deep-dive-idiomorph)

If you look at the details of the idiomorph demo you will see it's set up in a particular way: namely, the video cannot change the depth in the DOM at which it is placed, nor can any of the types of the parent nodes of the video change. This is a severe restriction on what sorts of UI changes idiomorph can handle. With the ability to reparent elements idiomorph could offer much better user experience, handling much more significant changes to the DOM without losing state such as video playback, input focus, etc.

Note that it's not only morphing algorithms like idiomorph that would benefit from this change: nearly any library that mutates the DOM would benefit from this ability. Even virtual DOM based libraries, when the rubber meets the road, need to update the actual DOM and move actual elements around. This change would benefit them tremendously.

Thank you for considering it!

[smaug----]

Anything else?

Add some complexity to selection/range: how to deal with Shadow DOM when the host moves around and selection is partially in shadow DOM?

[ydogandjiev]

This is a very exciting proposal! In the Microsoft Teams Platform, we extensively use iframes to host embedded apps in the Teams Web/Desktop Clients. When a user navigates away from an experience powered by one of these embedded apps and comes back to it later, we provide the ability for them to keep their iframe cached in the DOM (in a hidden state) and then re-show it later when it's needed again. To implement this functionality, we had to resort to creating the embedded app frames under the body of our page and absolute position them in the right place within our UX. This approach has lots of obvious disadvantages (e.g. breaks the accessibility tree, requires us to run a bounds synchronization loop, etc.) and the only reason we had to resort to it was because moving the iframe in the DOM would reload the embedded app from scratch thus negating any benefits of caching the frame. This proposal would allow us to implement a much more ideal iframe caching solution!

Note the location of the iframe in the DOM and its absolute positioning in this recording:
https://github.com/whatwg/dom/assets/3357245/7fd4d2a7-2c2d-4bed-9a78-9c60f26a42f4

[infogulch]

The WHATNOT meetings that occurred after this issue was created deferred discussion about the topic. I wonder what next steps would be needed to move this issue forward. The next meeting is on March 28 (#10215).

[noamr]

The WHATNOT meetings that occurred after this issue was created deferred discussion about the topic. I wonder what next steps would be needed to move this issue forward. The next meeting is on March 28 (#10215).

I hope we can get to it in the 28.3 WHATNOT. @domfarolino @past ?

[past]

It's already on the agenda, so if the interested parties are attending we will discuss this.