Make 1st array argument Extensible in Tag Functions

[Lonniebiz]

Unlike typical arrays, the array passed as the first argument to a tag function is not Extensible. Please read this:
https://stackoverflow.com/questions/53808065/array-argument-not-extensible-in-tag-function

Because it has never been extensible, making it extensible now, wouldn't create any backward compatibility issues.

Why make it extensible?

The fastest possible way to access processed-output, based on this array, is to add a property to the array itself that references the object that caches that processing.

Just like the array itself doesn't change on multiple tag functions calls, the processing done to that array also often does not change. Therefore, that processed output needs to be cached too. Right now, because this array is not extensible, you have to use a slower mechanism to do this caching: a Map object. Map objects are fast, but they are not faster than adding a property to the array and caching it there:

let cacheObject = someMapObject.get(aryOfStaticParts); // Is not faster than:
let cacheObject = aryOfStaticParts.cacheObject; // Faster, but can't do it (not extensible)

Making this array extensible shouldn't break current functionality. This specified prevention of extensibility is not essential to facilitating existing functionality.

In ecmascript, arrays were made extensible for a good reasons: flexibility and speed.

Some people frown upon adding properties to an array. But there are circumstances where doing so is the most performant way to accomplish a task. The code above is a perfect example of this.

The original designers of this language understood the flexibility and speed gained by making arrays extensible. I'd like new specifications to follow this precedent. I propose we change the specification to make this array extensible like every other default array in the language.

[ljharb]

Instead of mutating the array, could you keep a WeakMap, using the array as the key, and your "extra data" as the value?

[Lonniebiz]

Yes, you could. I actually settled for a regular Map object.. However, neither of those access the cacheObject faster than a property on the array itself.

[ljharb]

I believe if the array is mutable, it creates a cross-realm communications channel that violates per-compartment encapsulation use cases - which is why it must either be frozen, or newly generated for every call. @erights can speak more to this.

[erights]

Yes. There is precedent for the hell of having a literal expression repeatedly evaluate to the same mutable expression. In ES3 the RegExp literal /.../ evaluated to the same mutable RegExp object each time. We fixed this in ES5 despite the (huge in that case) risk of breaking old code.

For the template argument, we allowed the instance to be shared across evaluations only because it was transitively immutable. We had it be the same instance with the same identity exactly so that you could use a WeakMap (or a Map if you wish) to associate state with a particular one. To do so, you already needed to have shared mutability across evaluations, i.e., the map itself.

[Lonniebiz]

When an array is a key in a map, isn't it a reference and not a value? If so, it seems to me that you could add 1,000 properties to that array and it would not change its key and therefore not break any backwards compatibility. People currently using the map or weakmap caching method's code would not break.

If I'm wrong, and the array as key is by value, then I can see what you're talking about.

[erights]

It would break code that relies on the inability of other code to modify that value.

[Lonniebiz]

I was hoping you wouldn't think of that :)

But hey, like you said about changing that stuff with regex, we've made more invasive changes than this in the past.

[ljharb]

Note, though, that those types of risky changes are usually in the "more restrictive" direction, not "less restrictive".

[Lonniebiz]

If something was never extensible, then you know for sure that no one has ever put a property onto it.

Seems to me, that in this case, going "less restrictive" is less risky than the other way around (preventing extensibility on something that was formerly extensible).

[erights]

Less risky? In exactly the same way as making Object.freeze be a noop is less risky than making all objects frozen by default. Either is fatally risky. Existing code crucially relies on the restriction, and would become unsafe otherwise. Whereas making all objects frozen by default would at least fail safe.

[Lonniebiz]

Are you speaking about browser implementation code, user-land code, or both?

I'm sorry, I'm just trying to imagine a scenario in developer land where someone has written code that depends on the first argument of a tag function being immutable to the point it would break their application if you allowed the adding of a property to that array.

[erights]

both

https://www.youtube.com/watch?v=3ME7oHHQbuM
https://rawgit.com/Agoric/SES/master/demo/
https://www.youtube.com/watch?v=9Snbss_tawI&list=PLKr-mvz8uvUgybLg53lgXSeLOp4BiwvB2

[Lonniebiz]

Ok, that was educational, and after one viewing, I do not pretend to fully understand all the topics covered in those videos/articles.

However, I will tell you my fleeting thoughts after watching.

I'm cool with the specification providing developers the ability to enforce security on their own websites and node applications. If Salesforce wants to freeze and make immutable every object in their ecosystem, give them (with the specification) the ability to enforce just that.

But please don't force that upon me. When a new feature in javascript hands me an array, I want that array to act like every other array I've ever dealt with. Not some security-limited frozen array that causes me to have to use a Map object that is slower than accessing a property on the array directly. That kind of security-minded restriction needs to be off by default and people who believe "it actually secures anything" should have the ability to turn it on.

I'm creating an application from scratch using no 3rd party modules. What is this security limitation protecting me from doing besides optimizing the speed of my application?

[bakkot]

I think that avoiding non-obvious sources of persistent global-ish state is to the good of anyone trying to reason about programs, not just people interested in security work. And I don't think "insecure" is a good default for configurable things, though of course JS's history is very much insecure by default.

Separately, I want to push back on "setting a property on an array is faster than using it as a key in a map" as justification for changing the language's semantics. Those sorts of micro-optimizations tend to be highly tied to the engine you happen to be using right now. I would discourage trying to do that sort of thing in your code except in very rare circumstances, and as such I don't think the ability to make such micro-optimizations ought to guide the design of the language very much.

(Also, in this particular case, I bet it would be faster in current engines to put your cached information in a closed-over variable rather than as a property of the tag argument. Since #890 those are equivalent.)

Edit: rereading this I am realizing I neglected something important, which is, I get why it's surprising and frustrating that this one particular kind of array is weird. Just, I think the benefits - in particular avoiding new sources of global state - make it worth having the current behavior in spite of that.

[erights]

Yes. There is precedent for the hell of having a literal expression repeatedly evaluate to the same mutable expression. In ES3 the RegExp literal /.../ evaluated to the same mutable RegExp object each time. We fixed this in ES5 despite the (huge in that case) risk of breaking old code.

For the template argument, we allowed the instance to be shared across evaluations only because it was transitively immutable. We had it be the same instance with the same identity exactly so that you could use a WeakMap (or a Map if you wish) to associate state with a particular one. To do so, you already needed to have shared mutability across evaluations, i.e., the map itself.

[Lonniebiz]

When an array is a key in a map, isn't it a reference and not a value? If so, it seems to me that you could add 1,000 properties to that array and it would not change its key and therefore not break any backwards compatibility. People currently using the map or weakmap caching method's code would not break.

If I'm wrong, and the array as key is by value, then I can see what you're talking about.

[erights]

It would break code that relies on the inability of other code to modify that value.

[Lonniebiz]

I was hoping you wouldn't think of that :)

But hey, like you said about changing that stuff with regex, we've made more invasive changes than this in the past.

[ljharb]

Note, though, that those types of risky changes are usually in the "more restrictive" direction, not "less restrictive".

[Lonniebiz]

If something was never extensible, then you know for sure that no one has ever put a property onto it.

Seems to me, that in this case, going "less restrictive" is less risky than the other way around (preventing extensibility on something that was formerly extensible).

[erights]

Less risky? In exactly the same way as making Object.freeze be a noop is less risky than making all objects frozen by default. Either is fatally risky. Existing code crucially relies on the restriction, and would become unsafe otherwise. Whereas making all objects frozen by default would at least fail safe.

[Lonniebiz]

Are you speaking about browser implementation code, user-land code, or both?

I'm sorry, I'm just trying to imagine a scenario in developer land where someone has written code that depends on the first argument of a tag function being immutable to the point it would break their application if you allowed the adding of a property to that array.

[erights]

both

https://www.youtube.com/watch?v=3ME7oHHQbuM
https://rawgit.com/Agoric/SES/master/demo/
https://www.youtube.com/watch?v=9Snbss_tawI&list=PLKr-mvz8uvUgybLg53lgXSeLOp4BiwvB2

[Lonniebiz]

Ok, that was educational, and after one viewing, I do not pretend to fully understand all the topics covered in those videos/articles.

However, I will tell you my fleeting thoughts after watching.

I'm cool with the specification providing developers the ability to enforce security on their own websites and node applications. If Salesforce wants to freeze and make immutable every object in their ecosystem, give them (with the specification) the ability to enforce just that.

But please don't force that upon me. When a new feature in javascript hands me an array, I want that array to act like every other array I've ever dealt with. Not some security-limited frozen array that causes me to have to use a Map object that is slower than accessing a property on the array directly. That kind of security-minded restriction needs to be off by default and people who believe "it actually secures anything" should have the ability to turn it on.

I'm creating an application from scratch using no 3rd party modules. What is this security limitation protecting me from doing besides optimizing the speed of my application?

[bakkot]

I think that avoiding non-obvious sources of persistent global-ish state is to the good of anyone trying to reason about programs, not just people interested in security work. And I don't think "insecure" is a good default for configurable things, though of course JS's history is very much insecure by default.

Separately, I want to push back on "setting a property on an array is faster than using it as a key in a map" as justification for changing the language's semantics. Those sorts of micro-optimizations tend to be highly tied to the engine you happen to be using right now. I would discourage trying to do that sort of thing in your code except in very rare circumstances, and as such I don't think the ability to make such micro-optimizations ought to guide the design of the language very much.

(Also, in this particular case, I bet it would be faster in current engines to put your cached information in a closed-over variable rather than as a property of the tag argument. Since #890 those are equivalent.)

Edit: rereading this I am realizing I neglected something important, which is, I get why it's surprising and frustrating that this one particular kind of array is weird. Just, I think the benefits - in particular avoiding new sources of global state - make it worth having the current behavior in spite of that.

[Lonniebiz]

I'm not sure a closed-over variable would work in this special case because: One tag function can serve numerous tagged template literals and each TTL gives the (shared) tag function a different array as first argument. You could put a map object in a close-over, (which might be faster than where I put it -- a static class property) but nothing is faster than storing the cacheObject onto the vary array from which the cacheObject is "caching processing for".

Consider the fact that, in this case, the array would only have one property, while a map (instead) would have an entry for each tagged template literal served by the tag function. Accessing the property of an object, when that object has only one property, is pretty hard to beat in performance (no matter how optimized the map's engine implementation).

I'm persuaded by your point about making things secure by default. But please recognize, in this case, the array isn't merely non-extensible by default; the developer has no other choice regarding the matter. I'd like such security-minded presumptions, at most, be default and at least be optional (during creation).

[WebReflection]

It took forever in JS to have a way that wouldn't use objects as trash bin for any sort of randomly and leaking attached property that I'd be voting -1 forever to make the template tag first argument mutable by any mean.

Quite the opposite, I want it to be the fastest variable of them all, compiled right on top of the bare metal, so that everyone can read it, loop it, back and forward, 1000 times without any issue.

By specifications, Map or WeakMap are all it takes to relate a template array to anything, and that works, it's safe, it's not obtrusive when such array is passed around many functions for various reasons, and having it immutable is also a feature detection to distinguish between some TypeScript transpiled shenanigans and real-world specifications.

As pioneer of the usage of the template literal uniqueness (aka hyperHTML & viperHTML) I strongly wish this part of the specification, which has bitten me already once due changes, won't ever change again.

Please let's keep trying to make ECMAScript a standard people can trust, not wait 10 years before feeling comfortable they can use anything relatively new.

ES2015 isn't new, and Map or WeakMap neither (natively available since IE11 and every mobile browser).

Thank you in advance for not changing template literals again.

Best Regards

[Lonniebiz]

WebReflection is referring to this change that broke his code:
https://itnext.io/a-tiny-disastrous-ecmascript-change-fadc05c83e69

I brought this thread to his attention after reading that.

My thoughts on his comments is that if he wants something frozen, to freeze it himself. He should be able to do that, but it should be optional by the programmer (not mandatory from the standards body).

[littledan]

Now that we cache by Parse Node rather than template string (#890), does anything change with respect to cross-realm/membrane communication channel risks?