Bundles of JS modules should be able to express their dependency graph cheaply

[jyasskin]

If you have 2 modules:

A.mjs:

import "B"
// Lots of code.

B.mjs:

// Lots more code.

You have to download A and parse it to figure out that it depends on B. Then you have to download B and execute it before you can use nearly any of the bytes you downloaded for A.

Bundles should help speed this up, either by directly expressing the import dependency graph in a way that the JS interpreter will trust, or by letting the bytes of the resources interleave so they can parse and execute in the order they arrive.

[jyasskin]

@wycats wanted this too, IIRC.

[littledan]

Thanks for opening up this issue, @jyasskin . I'm really excited by this idea. Among complexities, in addition to describing the dependency structure, I'm wondering, how can you tell which Realm should the JavaScript module be run in? E.g., the bundle may include a page HTML with an iframe; I'm not sure if there are interactions with HTML modules as well.

One way that these semantics could be accessed today in HTML would be to insert many <script type=module> tags, in postorder for how the module graph would be traversed. If, in a bundled exchange package, the HTML file comes first, and then the scripts in the order that is in parallel with those script tags, then there would be somewhat similar observable behavior.

Including a built-in feature for this model of execution could be better than a bunch of tags in a few ways:

• Maybe most importantly, which @domenic marked as new behavior in his summary, is that the UA would be allowed to not yield to the event loop if the module is already fetched.
  • @nyaxt mentioned in Yielding to the event loop (in some way) between module evaluations whatwg/html#4400 (comment) that yielding to the event loop too many times could be difficult for browsers to make cheap.
  • However, @smaug---- expressed concern about optional yielding to the event loop in Yielding to the event loop (in some way) between module evaluations whatwg/html#4400 (comment).
• Fewer script tags, meaning shorter HTML to download, less DOM and possible events, and less need of tooling to generate the HTML. (I'm not sure how much overhead this actually is.)

Bottom-up loading could benefit JavaScript in terms of enabling bytecode generation to have access to dependencies having already been parsed. This isn't so necessary today, but may be helpful with a newer version of the the decorators proposal.

My guess is that bottom-up loading with optional yields to the event loop would not be web-incompatible (as @yoavweiss asked), since probably not many people are shipping big native module graphs yet, but it could violate JavaScript programmer expectations about atomicity of module loading, as @rniwa raised.

I'd be curious to figure out what parts of this might be possible to experiment with in bundler software today, and what parts require new web platform primitives. I imagine browsers might be able to make smarter decisions about how to interleave JavaScript execution and network access.

[littledan]

(Or, as the never-yielding complement to a bunch of script tags, you could have a single module script which imports all the modules that will eventually be used, in post-order. And of course tools could play around mixing these two together, but only statically.)

[jyasskin]

FWIW, I'm inclined to have folks write the single module script that imports everything in post-order, until we show that it costs enough more than a better solution to justify the complexity of the better solution.

[yoavweiss]

This assumed that the modules have no side-effects, right?

[jyasskin]

@yoavweiss It assumes they don't have syntax errors that are supposed to prevent side effects from happening, but I think it otherwise preserves side effects.

[littledan]

The never-yielding variant of generated import statements would preserve side effects (while defeating much of the benefit of the optimization)--the entire module graph is checked for errors before any of it runs. Otherwise, I agree with @jyasskin.

[rektide]

One possible strategy might be to use Link headers:

As a server, I am empowered to express relationships of my choosing via a Link header. If the ask in this ticket is for JS modules to express their dependency graph clearly, I a server might facilitate this ask my using a link header like:

Link: <http://yoyodyne.net/lib/dep.js>; rel=dependency; as=module

This is a declaration of a dependency of this script's resource. I don't know who or what would listen & detect this, or what they would do with this. But as a server, I can enumerate relations of this resource, for example rel=depenency, which would give the user-agent a way to understand dependencies without having to parse JS modules.

Link Preload might be related. Server admins might consider adding a preload relationship everywhere there is a dependency relationship, to let the browser know it will be expected.