[templates] Ensure that template instantiation actually improves the platform

[smaug----]

Before adding anything as complicated as https://github.com/w3c/webcomponents/blob/gh-pages/proposals/Template-Instantiation.md it is better to ensure it improves the platform.
I'm especially interested in the performance aspect.

If a polyfill could implement the proposal without being significantly slower, would there be need to add the API natively?

[jakearchibald]

Aside from the performance aspect, @surma is interesting it writing developer-facing documentation for this proposal, to lower the bar for feedback.

[JanMiksovsky]

I spent last week implementing a polyfill for a syntax variation of the template instantiation proposal, and also ported a number of web components from the Elix library over to that polyfill to see whether template instantiation would make creating such components easier.

Upon reflection, it does not appear that template instantiation really helps us.

The web components we create are typically more complex than can be addressed with mustache syntax. We use a mixin architecture to create our components; e.g., a complex carousel component of ours is currently built from ~20 different mixins, each contributing some aspect of programmatic or interaction behavior to the component class.

Significantly, in this mixin architecture, the component class that defines the template doesn't know what properties of sub-elements within that template will need to be modified by the mixins. Instead, a component asks itself (and therefore the mixins along its prototype chain) to build up a dictionary of all the updates that should be applied to the elements in its shadow tree. Then the component applies those updates.

Such separation of concerns means we cannot use the mustache syntax proposal, because that requires the complete set of attributes that will be modified to be expressed directly in the template.

One thing that might help would be a way for a component class to indicate that a collection of property updates be applied to a given element. Perhaps this would be something along the lines of React's spread syntax:

<div {...props}/>

We'd want to avoid introducing new syntax, so maybe we'd have a special property?

<div properties="{{props}}"></div>

I had a chance to talk about these experiments with @rniwa at Apple last week. During our conversation, I proposed adding something like the properties= idea above to Ryosuke. He suggested that, given our existing mixin architecture, what we'd really benefit from is something that lets us apply a collection of property updates to a tree of elements.

I've gone ahead and written up that idea as a proposal for bulk property updates. (Filed as a separate issue.) That includes a hypothetical applyPropertiesById method that would achieve what Ryosuke suggested. With something like that, we wouldn't need the template instantiation proposal.

In any event, for the time being, we don't see template instantiation helping our web components library.

[rniwa]

It's true that much (if not all) of what this API provides can be implemented in JS. For us, the main benefit of this proposal is that it paves a way to come up with a declarative syntax for custom elements. It also provides a useful mechanism for template libraries to update different parts of DOM easily. Finally, the default template processor provides a mechanism to easily create a template instance without having to import a third party library.

[mildred]

Hello, I'm new to this discussion, but I'd like to point out that there is no need for special syntax with curly braces to get useful things from templates. I'd like to point out two different template engines based on the same principles each that don't require any template syntax at all:

• https://github.com/pure/pure and https://pure-js.com/
• https://github.com/tmpvar/weld

They both work on the same principles: You define your markup with absolutely no special syntax, custom attributes or anything like that. Then, you define a JSON object on the side that goes with your markup that is telling how to instantiate the markup for a given input data structure.

What I really like from this approach is that you get a clear separation between the markup and the data structure you want to template. From the beginning, web standards is all about separation of concerns, and I believe this approach fits very nicely with that.

[dmitriid]

@rniwa

The main problem with template instantiation proposal is that it attempts to solve a symptom, not provide the cure for the cause.

And there are two causes:

• no declarative API for DOM APIs
• no data binding mechanism

As all compromises, template instantiation proposal solves both poorly.

JS. For us, the main benefit of this proposal is that it paves a way to come up with a declarative syntax for custom elements.
It also provides a useful mechanism for template libraries to update different parts of DOM easily.

This "syntax" already exists, and has been battle proven over several years on thousands of web sites. As is the ability to efficiently and easily update different parts of the DOM.

And it looks more or less like this:

h('div', 
   { 
        attrs: { className: "x" }, 
        props: { duration: y }, 
        on: { click: () => z()  }
   }, 
   [ h(...), h(...) ]
);

Yes. It's virtual DOM, popularised by React, and which exists as virtual-dom, hyperscript etc. There are differences in the APIs between these libraries, but they are quite superficial, and they more or less converged on the same set of principles:

• declarative API that is pure JS, no external dependencies, no templates, just functions
• the API is essentially f( tag, { props, attrs, events, lifecycle }, [children]) which is easy to reason about, easy to analyse, debug, optimise
• it solves about 100% of Web Component's data binding woes
• it solves the whole "we don't know how to provide a declarative API for WebComponents" because it also solves "we will provide a unified consistent API for all of HTML/DOM, not just WebComponents"
• it solves server-side rendering

Meanwhile templates... I don't know what actual issues do they solve? They introduce way more problems than it's worth:

• somehow only limited to template instantiation inside custom elements. Why not use them for the rest of the DOM?

• is a new templating language:

  • has no relation to the rest of the browser
  • needs a separate parser and execution model (not a small chunk of work in itself + an increased attack surface)
  • as all templating languages will be limited (so, limited control structures, limitations on what you can call from the host environment, poor and limited available standard library, if any)
  • only provides an imperative API to work with from JS that's no better and no easier than existing DOM APIs:

  // before
  var X = template.getElementById("#part");
  X.appendChild(document.createElement('span'));
  X.appendChild(document.createTextNode('hello'));
  
  // after
  var X = template.getRequiredTemplatePartSomehow();
  X.replace([document.createElement('span'), 'hello']);

  The vast majority of the web isn't even using DOM APIs with Web Components, and goes straight for .innerHtml (even lit-html does that with reckless abandon). Given that the template proposal doesn't really add any new easy-to-use APIs, it will still be easier to drop to .innerHTML or reach for any of the virtual dom libraries out there than use the templates.

In my opinion, the templating proposal has its value to show some the problems that people experience with Web Components, but can never be a solution. The solution lies in better, easier-to-use declarative DOM APIs.

[bahrus]

Hi @dmitriid

I don't follow why template instantiation could only be used inside a web component. It would be useful anytime there's repetitive markup that needs customizing in each instance. I encounter that scenario quite a bit, before and after web components.

The fundamental issue templates are trying to solve, as I understand it, is that repeatedly cloning a template for large chunks of easy to parse html is faster than making lots of appendChild or innerHTML calls, which can only be done after the expensive job of parsing the JavaScript. This is an empirical claim. Are you claiming otherwise? It would be great to show your counter-factual results.

HTML templates also feel much more "declarative" to me. Templates are inert, containing a data format (xml-ish) so they can be loaded quickly with no side effects. hyperscript may also have no side effects for simple examples (other than rendering the html of course), but you can evaluate any function you want during the processing, which could have unexpected side effects. To be fair, some aspects of the template instantiation proposal (which looks quite different from how you are describing it, with your before and after -- are we looking at the same proposal?) may also allow for functions with side effects, but I could be wrong.

For the record, I'm not opposed to introducing an h function into the api, if it fulfills some useful purpose, but I fail to see how it is easier to use than tagged template literals. Could you elaborate? The performance numbers I've seen comparing lit-html and hyperHTML, compared to react (and even preact) make me wonder what your objections are?

[dmitriid]

@bahrus

I do apologise if I sound terse or rude in the text below, as I'm writing this rather quickly in a spare moment (I wanted to acknowledge your response quickly, and not have you wait for a day or two).

I don't follow why template instantiation could only be used inside a web component. It would be useful anytime there's repetitive markup that needs customizing in each instance.

The whole discussion is mostly in the context of Custom Elements and Shadow DOM (see 2. Use Cases). The proposal itself is only limited to <template> elements.

That's why in my mind it was only limited to custom elements.

However, true, you can use them elsewhere:

rniwa = {name: "R. Niwa", email: "rniwa@webkit.org"};
document.body.appendChild(contactTemplate.createInstance(rniwa));

This does still leave the question of how to more complex/nested templates where parts of a template are defined in other templates etc.

but you can evaluate any function you want during the processing, which could have unexpected side effects.

That is, side effects that are desired by the developer using it ;). There's no way to use a template that has this:

{{foreach items}}
   <li class={{class}} data-value={{value}}>{{label}}</li>
{{/foreach}}

without first calling arbitrary functions to create these items and binding them to the template. And since the proposed way of creating such things is the same old DOM API with hardly any improvements, I fail to see the improvement.

And as you correctly mentioned, templates will need to be able to call arbitrary functions.

I fail to see how it is easier to use than tagged template literals. Could you elaborate? The performance numbers I've seen comparing lit-html and hyperHTML, compared to react (and even preact) make me wonder what your objections are?

The only reason lit-html works as it does is that dozens (hundreds?) of engineers spent hundreds of hours optimising to things that are frequently used and abused in JS:

• string processing and RegExps
• .innerHtml

More or less the only thing that lit-html does is parse a string at runtime, with regexps, concatenate it into an opaque string blob, and dump it into browser via .innerHtml, and let browser deal with it. (I have a separate rant on tagged template literals elsewhere).

It's not a good thing. It's a very bad thing, it only happens to work fast enough because browsers have had decades to optimise this (and ten years ago using .innerHtml was considered extremely bad practice see e.g. this StackOverflow comment, and even now .innerHtml is not optimised enough for certain cases, and there are security considerations).

Meanwhile declarative DOM/virtual DOM libraries have to recreate the entire DOM model in memory, and manually diff it against the browser DOM.

The solution to all that (and to template instantiation) is definitely not, in my opinion:

• a limited templating language that can only be used in <template> elements, and whose actual API is only marginally better than the rest of DOM APIs (and introduces a host of other problems, some of which
• concatenating strings at runtime and dumping them into the DOM via .innerHTML

A declarative API (with, hopefully, browser-native DOM-diffing) solves a lot of the problems:

• data binding (it just becomes function calls with your data/parameters)
• efficient DOM updates (you only regenerate parts of the tree that change)
• instantiation (it's just a function call, and many lifecycle methods to trigger it that are already there)

It may/will still be awkward to use, obviously. The next best thing, IMO, would be a standard/declarative way to create a DOM AST that you can pass to the browser. Virtual DOM libs, in essence, do that already. To quote Dan Abramov:

// JSX is a syntax sugar for these objects.
// <dialog>
//   <button className="blue" />
//   <button className="red" />
// </dialog>
{
  type: 'dialog',
  props: {
    children: [{
      type: 'button',
      props: { className: 'blue' }
    }, {
      type: 'button',
      props: { className: 'red' }
    }]
  }
}

Unfortunately, we cannot generate this AST for the browser and let it deal with it efficiently. We have to fall back to .innerHtml or hundreds of lines of tedious imperative code (or use libs/frameworks/wrappers).

With a natively supported declarative description of the DOM/AST you still have a low-level primitive that libs/frameworks can make even easier to use, but you can also trivially use it in vanilla JS code.

I do hope I made sense in the ramblings above :)

[bahrus]

Thanks, @dmitriid, for your civil and non-rambling response :-)

I agree templates are a key part of the web component stack, and in retrospect you were replying to a message arguing why an (apparently) small library like what template instantiation would (apparently) entail should be built in to the platform, and the argument was made that it would benefit declarative custom elements.

But more generally, I for one have taken the liberty of using templates without web components on a number of occasions, and I don't think I'm alone.

I actually agree with you that the platform would benefit from some helper functions for those scenarios where a programmatic api is needed -- one that improves upon Object.assign, one that is specifically tailored for setting properties / attributes / events on a DOM element, i.e. a similar purpose to what you are laying out.

For example, I've been toying with a function I call "decorate", which I modeled after Vue/Polymer 1, but now realize, thanks to your bringing it to my attention, is quite similar to h.

The difference is that this decorate function is applied to an existing element, coming from a template or existing DOM tree, rather than being only useful for generating the HTML data structure itself.

I do think, given the wide range of libraries doing something similar, having something built in to the platform would be useful. Just my two cents. But so would template instantiation. I don't see why one precludes the other.

If a template api/syntax has built in functions to do certain things (like for-each), i.e. officially sanctioned functions, that is quite different from saying you can use any user-defined function you want. For a concrete example of what I'm getting at, I need to change the subject slightly: Take the github developers who are managing the github web site-. They are defining web components, which can enhance the markdown vocabulary. I recently realized I can use their custom elements in my markdown! Github only trusts that because they trust their own web components. They won't allow us to use arbitrary web components, for fairly obvious reasons.

I'm not saying user-defined functions must be forbidden in template languages, only that it is a significant line one is crossing, one which could be use to separate "non-declarative" vs "declarative" in a coherent (I think) way.

I suspect the author of hyperHTML would take issue with the statement that it required dozens or hundreds of engineers to match the performance of h based libraries :-).

Thanks for the link to your critique of tagged template literals. I'll take a look.

[WebReflection]

I suspect the author of hyperHTML would take issue with the statement that it required dozens or hundreds of engineers to match the performance of h based libraries

I think @dmitriid was referring to the fact these libraries (lit, lighter, or hyperHTML) are fast only because the primitives used have been made fast by browsers engineers.

I also think there's no shame in knowing, and using, fast primitives to deliver better UX, and that's naturally the goal of any performance oriented abstraction anyway 👋

[bahrus]

@dmitriid, is that what you meant? If so, apologies for misinterpreting.

I guess I was thrown by the use of the word "to" in:

The only reason lit-html works as it does is that dozens (hundreds?) of engineers spent hundreds of hours optimising to things that are frequently used and abused in JS:

followed by his comment on what the browser engineers have done:

it only happens to work fast enough because browsers have had decades to optimise this.

I.e hundreds of lit-html engineers (I can't seem to locate his reference to hyperHTML, which line is that?) used bad practices, which happened to not matter because of the decades spent by browser engineers optimizing on those bad practices.

Apologies for my lack of reading comprehension.

I agree with you -- lit-html, lit, hyperHTML are fast and easy to use, great libraries, because they were built and optimized by a handful of dedicated and smart engineers, built on fast primitives built by great browser engineering teams. Somehow I didn't quite find that sentiment shared by @dmitriid, but what do I know? @dmitriid, we're all in agreement?

[Jamesernator]

@dmitriid Neither template instantiation or lit-html use .innerHTML to update the DOM. This seems to be some FUD around how template instantiation works but template instantiation and lit-html actually work more efficiently than a virtual DOM by holding references to part of the DOM that have template parts. In lit-html's case, it only creates the initial template using .innerHTML, to actually update the DOM it inserts nodes directly into the correct location (and similar for attributes).

For example consider this template:

<template id="exampleTemplate">
  <span title="{{foo}}">This is a sample {{bar}} with instantiation</span>
  <span>Also {{foo}}</span>
</template>

when we create an instance of the template a set of references are created to the elements that have that name:

{
  foo: [
    AttributeTemplatePart {
      el: /* ref to <span title="{{foo}}"> element */
      attrName: 'title',
    },
    NodeTemplatePart {
      previousNode: /* The previous text node before {{foo}}: "Also" */
      nextNode: /* The text node after {{foo}} */
    },
  ],
  bar: [
    NodeTemplatePart {
      previousNode: /* The previous text node before {{bar}}: "This is a sample " */
      nextNode: /* The next text node after {{bar}}: " with instantiation"
    }
  ],
}

now when we call something like instance.update({ foo: 'banana', bar: 'cabbage' }) all .update needs to do is something like this:

function update(data) {
  for (const [key, value] of Object.entries(data)) {
    for (const templatePart of this._templateParts[key]) {
      templatePart.update(value)
    }
    this._templateParts[key].update(value)
  }
}

this is way more efficient than maintaining a virtual copy of the DOM as we simply implement AttributeTemplatePart and NodeTemplatePart so that they go directly to their location in the DOM and update the value.

e.g. NodeTemplatePart could be implemented like this:

class NodeTemplatePart {
  constructor(previousNode, nextNode) {
    this._previousNode = previousNode
    this._nextNode = nextNode
  }

  // This is overly simplified and assumes there's a node both before and after
  // the {{curlies}}, a real implementation would hold a reference to the parent element
  // as well and if there's no previousNode/nextNode it'd just replace the whole contents
  update(values) {
    if (typeof values === 'string') {
      values = [new Text(values)]
    }
    while (this._previousNode.nextNode !== this._nextNode) {
      this._previousNode.nextNode.remove()
    }

    for (const value of values) {
      this._nextNode.parentNode.insertBefore(value, this._nextNode)
    }
  }
}

[WebReflection]

Just for documentation and clarification sake, everything @Jamesernator said is the exact same for both lighterhtml and hyperHTML, based indeed on the same domdiff library, which uses innerHTML once per unique template tag, on an offline template element, and update the related DOM when needed, it never uses innerHTML again (unless explicitly required by the user, but that's another story)