Rehydration #549

wycats · 2017-06-15T00:55:43Z

This commit introduces a "rehydration" mode to the Glimmer VM.

-- What is Rehydration?

The rehydration feature means that the Glimmer VM can take a DOM tree
created using Server Side Rendering (SSR) and use it as the starting
point for the append pass.

Rehydration is optimized for server-rendered DOMs that will not need
many changes in the client, but it is also capable of making targeted
repairs to the DOM:

if a single text node's value changes between the server and the
client, the text node itself is repaired.
if the contents of a {{{}}} (or SafeString) change, only the
bounds of that fragment of HTML are blown away and recreated.
if a mismatch is found, only the remainder of the current element
is cleared.

What this means in practice is that rehydration repairs problems
in a relatively targeted way, and doesn't blows away more content
than the contents of the current element when a mismatch occurs.

Near-term work will narrow the scope of repairs further, so that
mismatches inside of blocks only clear out the remainder of the
content of the block.

-- What Changes Were Needed?

Previously, code in the append pass was permitted to make DOM
changes at any point, so long as they made the changes through
the stateless "DOM Helper" abstraction, which ensured that
code didn't inadvertantly rely upon environment-specific
behaviors. This includes both browser quirks (including the
most recent version of Safari), and restricting Glimmer to
the subset of DOM used in SimpleDOM, the library that is
responsible for emulating the DOM in our current Server
Side Rendering implementation.

Rehydration works by replacing attempts to create DOM nodes
with a check for whether the node that Glimmer is trying to
create is already present. It maintains a "cursor" against
the unhydrated DOM, and when Glimmer attempts to create an
element, it first checks to see whether the candidate node
matches the node that Glimmer is trying to create.

For example, if Glimmer wants to create a text node,
the rehydrator checks to see if the node under the cursor
is a text node. If it is a text node, it updates its
contents if necessary. If not, it begins a coarser-grained
repair (which, as described above, is never worse than
clearing out the rest of the DOM for the current element).

This requires that the core DOM operations not only go
through a stateless abstraction (DOM Helper), but also
have a choke-point through a stateful builder that can
maintain the candidate cursor and initiate repairs.

Most of this commit restructures code so that DOM operations during
the append pass go throug the central ElementBuilder in all cases.

Part of this process involved restructuring the code that manages
dynamic content and dynamic attributes to simplify them and make
them a better fit for ensuring that all DOM operations in the
append pass go through the ElementBuilder.

-- Serialize Mode

This commit also introduces a dedicated "serialize" mode that
server-side renderers should use to generate the DOM.

The serialize mode is reponsible for inserting additional markers
into the DOM to serialize boundaries that otherwise serialize
incorrectly.

For example, it inserts a comment node between two adjacent
text nodes so that they remain separated when rehydrating.

The serialize mode does not prescribe a DOM implementation;
any DOM that is compatible with the well-defined SimpleDOM
subset will work (including a real browser's DOM, JSDOM,
or SimpleDOM); it just ensures that the created DOM will
round-trip through a string of HTML.

Importantly, the rehydrator mode assumes that the server-
provided DOM was created in serialize mode.

-- Future Work: Attributes and Properties

Today, when you write something like <div title={{value}}>,
Glimmer attempts to use DOM properties if possible. This
works, more or less, because the DOM automatically reflects
properties to attributes in most cases.

This means that if you write <img src={{url}} /> in a template,
we will set the src property, which the DOM automatically
reflects onto the src attribute.

Using properties where possible as opposed to attributes in
all cases addresses a number of use cases. For example,

<select value={{someValue}}> will select the correct
<option> automatically
<div onclick={{action foo}}> "just works" to set
event handlers.
<textarea value={{someValue}}> updates the inside of
the textarea instead of requiring you to deal with
the text node contents of the textarea.

We intend to move to all-attributes-all-the-time in the future, but
we need to address these use-cases. There are various approaches
under consideration (including a few special-cases, element
modifiers, and an alternate sigil), but they are out of scope
for this PR.

In order to address this gap for now, this PR works around the slight
semantic inconsistency. When the rehydrator pushes an element, it
snapshots the list of all attributes that are already present on
the element as a list of candidates for attribute removal. When
Glimmer attempts to set an attribute or property on the element,
the rehydrator removes it from the list of candidates for attribute
removal.

When an element's attributes are "flushed" (a step in the Glimmer
VM), the rehydrator removes any attributes that it didn't see during
the append step.

This should work in the vast majority of cases because:

First, it doesn't stop properties from being set, so any case where
properties were set before continue to be set now.

Second, if an attribute was present and the same-named property was
set in the client, the attribute will remain in the DOM. This
can only go wrong if the application was relying on a property
being set that didn't set the same-named attribute. This
is very rare; one example might be if an app was relying on using
form.reset() to reset a field back to '', relying on the
fact that <input value={{someValue}}> doesn't actually
set the value attribute.

This is very unlikely because empirically very few people use
.reset() in Ember or Glimmer, which is why using a property
instead of an attribute here worked in the first place.

In short, if someone is relying on the fact that an element's
attribute actually sets a property also relies on the fact that
it doesn't set an attribute, the current workaround will repair
inconsistently. This should be very rare and something we can
address next.

chancancode · 2017-06-15T18:26:39Z

packages/@glimmer/runtime/lib/dom/helper.ts

@@ -125,7 +125,7 @@ export namespace DOM {
      return this.document.createElementNS(namespace, tag);
    }

-    setAttribute(element: Element, name: string, value: string, namespace?: string) {
+    setAttribute(element: Element, name: string, value: string, namespace?: Option<string>) {


namespace : Option<string> = null?

chancancode · 2017-06-15T18:27:36Z

packages/@glimmer/runtime/lib/environment.ts

+// import {
+//   defaultManagers,
+//   AttributeManager
+// } from './dom/attribute-managers';


chancancode · 2017-06-15T18:27:56Z

packages/@glimmer/runtime/lib/environment.ts

@@ -455,8 +456,8 @@ export abstract class Environment {
    transaction.commit();
  }

-  attributeFor(element: Simple.Element, attr: string, isTrusting: boolean, namespace?: string): AttributeManager {
-    return defaultManagers(element, attr, isTrusting, namespace === undefined ? null : namespace);
+  attributeFor(element: Simple.Element, attr: string, isTrusting: boolean, namespace: Option<string>): DynamicAttributeFactory {


namespace: Options<string> = null for consistency?

chancancode · 2017-06-15T18:28:39Z

packages/@glimmer/runtime/lib/scanner.ts

+      buffer.push(statement);
+      this.state = LayoutState.AfterFlush;
+      return;
+    }


✂️ this and modify the resulting failing test

chancancode · 2017-06-15T18:29:58Z

packages/@glimmer/runtime/lib/scanner.ts

+      this.processStatement(attrs[i], buffer);
+    }
+
+    this.processStatement([ Ops.FlushElement ], buffer);


we might want to extract these into const since they just get copied into another array anyway

(there are a bunch in this file)

chancancode · 2017-06-15T18:39:32Z

packages/@glimmer/runtime/lib/vm/content/text.ts

+      normalized = value;
+    } else {
+      normalized = String(value);
+    }


normalizeStringValue?

chancancode · 2017-06-15T18:42:37Z

packages/@glimmer/runtime/lib/vm/serialize-builder.ts

+  __appendHTML(html: string): Bounds {
+    let first = this.__appendComment('%glimmer%');
+    super.__appendHTML(html);
+    let last = this.__appendComment('%glimmer%');


This probably wants to be a matching pair, like  and  perhaps. If we think it's not important to have a matching pair for this case then an empty comment is just fine.

chancancode · 2017-06-15T18:43:32Z

packages/@glimmer/runtime/lib/vm/serialize-builder.ts

+    return bounds(this.element, first, last);
+  }
+
+  __appendText(string: string): Simple.Text {


Does not handle empty text nodes

chancancode · 2017-06-15T18:43:54Z

packages/@glimmer/runtime/lib/vm/serialize-builder.ts

+    let current = currentNode(this);
+
+    if (current && current.nodeType === Node.TEXT_NODE) {
+      this.__appendComment('%sep%');


Could just use an empty comment here

chancancode · 2017-06-15T18:44:33Z

tsconfig.json

@@ -16,7 +16,7 @@
    "noImplicitAny": true,
    "suppressImplicitAnyIndexErrors": true,
    "noUnusedLocals": true,
-    "noUnusedParameters": true,
+    // "noUnusedParameters": true,


jeffwhelpley · 2017-06-16T00:40:31Z

We have considered something similar for Angular in the past and are still experimenting with a couple diff options. One thing I am curious about. Do you have performance metrics for larger pages comparing full view re-rendering vs this rehydration approach?

PatrickJS · 2017-06-16T18:37:28Z

great write up 👍

sclatter · 2017-06-17T16:32:39Z

Super, @wycats !

This commit introduces a "rehydration" mode to the Glimmer VM. -- What is Rehydration? The rehydration feature means that the Glimmer VM can take a DOM tree created using Server Side Rendering (SSR) and use it as the starting point for the append pass. Rehydration is optimized for server-rendered DOMs that will not need many changes in the client, but it is also capable of making targeted repairs to the DOM: - if a single text node's value changes between the server and the client, the text node itself is repaired. - if the contents of a `{{{}}}` (or SafeString) change, only the bounds of that fragment of HTML are blown away and recreated. - if a mismatch is found, only the remainder of the current element is cleared. What this means in practice is that rehydration repairs problems in a relatively targeted way, and doesn't blows away more content than the contents of the current element when a mismatch occurs. Near-term work will narrow the scope of repairs further, so that mismatches inside of blocks only clear out the remainder of the content of the block. -- What Changes Were Needed? Previously, code in the append pass was permitted to make DOM changes at any point, so long as they made the changes through the stateless "DOM Helper" abstraction, which ensured that code didn't inadvertantly rely upon environment-specific behaviors. This includes both browser quirks (including the most recent version of Safari), and restricting Glimmer to the subset of DOM used in `SimpleDOM`, the library that is responsible for emulating the DOM in our current Server Side Rendering implementation. Rehydration works by replacing attempts to create DOM nodes with a check for whether the node that Glimmer is trying to create is already present. It maintains a "cursor" against the unhydrated DOM, and when Glimmer attempts to create an element, it first checks to see whether the candidate node matches the node that Glimmer is trying to create. For example, if Glimmer wants to create a text node, the rehydrator checks to see if the node under the cursor is a text node. If it is a text node, it updates its contents if necessary. If not, it begins a coarser-grained repair (which, as described above, is never worse than clearing out the rest of the DOM for the current element). This requires that the core DOM operations not only go through a stateless abstraction (DOM Helper), but also have a choke-point through a stateful builder that can maintain the candidate cursor and initiate repairs. Most of this commit restructures code so that DOM operations during the append pass go throug the central ElementBuilder in all cases. Part of this process involved restructuring the code that manages dynamic content and dynamic attributes to simplify them and make them a better fit for ensuring that all DOM operations in the append pass go through the ElementBuilder. -- Serialize Mode This commit also introduces a dedicated "serialize" mode that server-side renderers should use to generate the DOM. The serialize mode is reponsible for inserting additional markers into the DOM to serialize boundaries that otherwise serialize incorrectly. For example, it inserts a comment node between two adjacent text nodes so that they remain separated when rehydrating. The serialize mode does not prescribe a DOM implementation; any DOM that is compatible with the well-defined SimpleDOM subset will work (including a real browser's DOM, JSDOM, or SimpleDOM); it just ensures that the created DOM will round-trip through a string of HTML. Importantly, the rehydrator mode assumes that the server- provided DOM was created in serialize mode. -- Future Work: Attributes and Properties Today, when you write something like `<div title={{value}}>`, Glimmer attempts to use DOM properties if possible. This works, more or less, because the DOM automatically reflects properties to attributes in most cases. This means that if you write `<img src={{url}} />` in a template, we will set the `src` property, which the DOM automatically reflects onto the `src` attribute. Using properties where possible as opposed to attributes in all cases addresses a number of use cases. For example, - `<select value={{someValue}}>` will select the correct `<option>` automatically - `<div onclick={{action foo}}>` "just works" to set event handlers. - `<textarea value={{someValue}}>` updates the inside of the textarea instead of requiring you to deal with the text node contents of the textarea. We intend to move to all-attributes-all-the-time in the future, but we need to address these use-cases. There are various approaches under consideration (including a few special-cases, element modifiers, and an alternate sigil), but they are out of scope for this PR. In order to address this gap for now, this PR works around the slight semantic inconsistency. When the rehydrator pushes an element, it snapshots the list of all attributes that are already present on the element as a list of candidates for attribute removal. When Glimmer attempts to set an attribute **or property** on the element, the rehydrator removes it from the list of candidates for attribute removal. When an element's attributes are "flushed" (a step in the Glimmer VM), the rehydrator removes any attributes that it didn't see during the append step. This should work in the vast majority of cases because: First, it doesn't stop properties from being set, so any case where properties were set before continue to be set now. Second, if an attribute was present and the same-named property was set in the client, the attribute will remain in the DOM. This can only go wrong if the application was relying on a property being set that **didn't** set the same-named attribute. This is very rare; one example might be if an app was relying on using `form.reset()` to reset a field back to `''`, relying on the fact that `<input value={{someValue}}>` doesn't **actually** set the value attribute. This is very unlikely because empirically very few people use `.reset()` in Ember or Glimmer, which is **why using a property instead of an attribute here worked in the first place**. In short, if someone is relying on the fact that an element's attribute actually sets a property also **relies on the fact that it doesn't set an attribute,** the current workaround will repair inconsistently. This should be very rare and something we can address next.

This reverts commit 830d0a5.

This reverts commit 96584b4.

This reverts commit cfd524a.

rwjblue · 2017-06-19T18:36:15Z

packages/@glimmer/runtime/test/new-initial-render-test.ts

+    return callback();
+  }
+
+  protected assertStableNodes({ except: _except }: { except: Set<Node> | Node | Node[] } = { except: new Set() }) {


We still support browsers without native Set (and we do not require polyfills). We need to replace this with something else...

I have this fixed locally. All we need to do is uniq the array.

ming-codes · 2017-08-03T08:10:24Z

How will this affect DOM nodes that are moved/reordered without Glimmer knowing? Will rehydration just blow that away and rebuild?

chadhietala · 2017-08-04T14:33:23Z

When we are in rehydration mode we scan over top of the SSR'd HTML. When we detect something is not correct we clear to the end of the current bounds and client-side render it. This means that we only throw out subsections of the SSR'd page when they don't match as opposed to chucking the whole document out and starting over.

jeffwhelpley · 2017-08-04T15:04:35Z

@chadhietala how is hydration matching up which node in the server view corresponds to a particular client side component? Is it based off position in the DOM tree?

wycats · 2017-08-04T17:33:07Z

@jeffwhelpley it's based on position in the DOM + some additional markers to track block start and end.

In Glimmer, a template like this:

<div>{{hello}}</div>

<some-component />

<some-other-component>{{contents}}</some-other-component>

is actually highly static. The output from the server-side serializer will place block boundaries around the dynamic parts that can contain more than one node (like components), but the nodes themselves cannot move around. As a result, "position in DOM" is a relatively precise way to identify nodes in Glimmer.

PatrickJS · 2017-08-04T18:03:06Z

I see, so similar to Resource Hints you're providing hints to the client compiler from the server compiler to create the correct structure.

chancancode requested changes Jun 15, 2017

View reviewed changes

wycats and others added 10 commits June 19, 2017 11:15

Update TypeScript

d354d02

Try to update to an npm-based build

bc4993a

Run npm i again

a50be2c

Put npm i in the right place

0dfe856

Start writing mismatch tests and improve the rehydrator

5b63c03

Revert "Put npm i in the right place"

4ea82f2

This reverts commit 830d0a5.

Revert "Run npm i again"

5c20097

This reverts commit 96584b4.

Revert "Try to update to an npm-based build"

7581187

This reverts commit cfd524a.

Fix TSLint errors

b1ef501

chadhietala force-pushed the rehydration branch from 1325aec to b1ef501 Compare June 19, 2017 18:23

rwjblue reviewed Jun 19, 2017

View reviewed changes

chadhietala added 2 commits June 19, 2017 13:30

Make Phantom happy

9c690dd

Fix IE null value serialization

bb21d74

chadhietala force-pushed the rehydration branch from ed47f45 to bb21d74 Compare June 20, 2017 04:27

chadhietala added 3 commits June 21, 2017 09:39

Fix IE

876eda9

Fix type errors and build issue

c05bc54

Address some of Godfrey's comments

b64e1b6

chadhietala force-pushed the rehydration branch from b58d4d7 to b64e1b6 Compare June 21, 2017 19:55

chadhietala merged commit cdd69e2 into master Jun 21, 2017

locks deleted the rehydration branch August 3, 2017 08:52

Turbo87 added the enhancement label Oct 22, 2017

IvanSanchez mentioned this pull request Sep 6, 2018

Wrap references to window to prevent them from erroring in node.js environment Leaflet/Leaflet#6332

Closed

lifeart mentioned this pull request Feb 17, 2024

[Disco]: recommending on{event} instead of on-modifier ember-learn/guides-source#2009

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rehydration #549

Rehydration #549

wycats commented Jun 15, 2017

chancancode Jun 15, 2017

chancancode Jun 15, 2017

chancancode Jun 15, 2017

chancancode Jun 15, 2017

chancancode Jun 15, 2017

chancancode Jun 15, 2017

chancancode Jun 15, 2017

chancancode Jun 15, 2017

chancancode Jun 15, 2017

chancancode Jun 15, 2017

chancancode Jun 15, 2017

jeffwhelpley commented Jun 16, 2017

PatrickJS commented Jun 16, 2017

sclatter commented Jun 17, 2017

rwjblue Jun 19, 2017

chadhietala Jun 19, 2017

ming-codes commented Aug 3, 2017

chadhietala commented Aug 4, 2017 •

edited

Loading

jeffwhelpley commented Aug 4, 2017

wycats commented Aug 4, 2017

PatrickJS commented Aug 4, 2017

Rehydration #549

Rehydration #549

Conversation

wycats commented Jun 15, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jeffwhelpley commented Jun 16, 2017

PatrickJS commented Jun 16, 2017

sclatter commented Jun 17, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ming-codes commented Aug 3, 2017

chadhietala commented Aug 4, 2017 • edited Loading

jeffwhelpley commented Aug 4, 2017

wycats commented Aug 4, 2017

PatrickJS commented Aug 4, 2017

chadhietala commented Aug 4, 2017 •

edited

Loading