Peeling out import()

[domenic]

Hi all,

We're getting lots of interest from multiple corners in providing a way to add async loading to the module support specced for browsers. I was wondering if it'd be OK if I started work on a TC39-side import() spec? It would absorb some of the responsibilities of this repo, so I wanted to check here first.

The basic ideas I have so far:

• import("./specifier.js") returns a promise for a module namespace object
• import() works in both scripts and modules
  • At first I thought only modules was appropriate, but it turns out that we can make this work in scripts without much extra work, and this is key for enabling lots of important interop scenarios.
• import() first calls HostFetchModule(currentScriptOrModule, specifier)
  • this returns a promise for undefined, or is maybe just "spec asynchronous" in some way
  • In HTML, this would be defined to first call resolve a module specifier using specifier + currentScriptOrModule, then call fetch a module script tree with the resulting URL and currentScriptOrModule's settings object.
• import() then calls HostResolveImportedModule(currentScriptOrModule, specifier)
  • We need to upgrade HostResolveImportedModule to also work for scripts as the first argument, not just modules
  • HTML needs to start storing a script URL inside the HTML classic script, just like it does for the HTML module script. Then we can retrieve the HTML classic script using argument.[[HostDefined]] and do the same things as today.
• import() then returns the resulting module record's [[Namespace]] object.

I think this would be a pretty small spec. It would, of course, have room for expansion in the future to support all the loader hooks, e.g. Loader could define a HostFetchModule for host environments that implement the Loader spec that goes through all the hooks, similar to what it does with HostResolveImportedModule today. But in the meantime it would give browsers and Node something they could implement.

Let me know what you think! If you're OK with me taking on this effort, I'd plan to have some spec text ready in time for the next TC39.

/cc @bmeck

[stuartpb]

I'm just coming in from nodejs/node-eps#39 - are we talking about defining a window.import (or whatever it'd be called in non-browser global contexts), which would be unambiguous from a keyword invocation of import solely due to the lack of parentheses on whatever's following the keyword?

EDIT: Taking another look at the spec, I guess import getting treated like a name rather than a keyword depending on surrounding syntax is already a thing, for import.context &c. Was there some revision to the syntax rules for names that are the same as keywords circa ES2015 (making other constructions like new.target valid as well)? I know ES5 opened the door to property identifiers like .get, but this notion of base names that are the same as keywords is something I've only seen in the last couple of years.

[bmeck]

@stuartpb keyword as a function, not a global.

[wrt import.context] I have been investigating something like import("js:context") instead of another meta-property.

[concavelenz]

What am I missing? How are a script and a module distinguished from each other? Can I see an example? +1 otherwise.

[ljharb]

@concavelenz they're different parsing goals. it's detailed in the spec.

[concavelenz]

@ljharb Yes, they are different parse goal which means you need to say which you intend. As the standard is now, any valid strict mode script is a valid module. So I'm asking is how do you state which you mean?

[ljharb]

At the moment, you communicate that out-of-band. <script type="module">, for example.

[concavelenz]

@ljharb Exactly. So what is the proposal here?

[domenic]

The OP is the proposal. What is unclear about it? Your confusion about how you distinguish scripts and modules seems unrelated to the OP.

[bmeck]

https://whatwg.github.io/loader/#loader-import ~= the thing being pulled out

[caridy]

@domenic in principle, I'm supportive of this, but I will like to dig in more on the idea of enabling import() in scripts as well. The interop issue could be done in user-land by creating a <script type=module> in browsers, and in node we know what needs to happen, isn't that enough to guarantee that a script can import a module and wait for it to finish? or the real issue is accessing the exports of the fetched module? My main concern is the changes needed in HostResolveImportedModule to support scripts, this happens to be the single most used abstract operation of the entire module spec.

Additionally, how will this play with the other effort that we (@benjamn and I) will be presenting (hopefully) for the await import statements in modules as discussed in the previous meeting?

[bmeck]

in node we know what needs to happen

The behavior of importing differs pretty drastically between require and what browsers are aiming for:

1. specifiers are URL parsed / Node import path resolution differs from require slightly
2. unwind stack prior to any work/eval (sync access of namespace == undefined/TDZ for most things)
3. needs to have an await-able Promise that can be differentiated from CJS exporting a Promise

I don't really have any clue on the spec for await import as a 2 keyword statement so I will hold off until discussions.

[concavelenz]

I'm not confused about how to distinguish between scripts or modules. I simply misread: "import() works in both scripts and modules" as "import() works for both scripts and modules" and didn't see that was described in the steps.

[domenic]

@caridy great!

You can indeed work around these issues somewhat in browsers, using something like https://gist.github.com/domenic/fd84ee5f4e2dc0278ab1#file-import-module-current-js. However, it uses a URL instead of a module specifier, and will be relative to the document's URL instead of the script's URL. (In addition to it being super-ugly, modifying the DOM and creating a global temporary variable.) I think giving people a first-class solution would make a lot more sense, especially since there really is no cost to doing so.

You mentioned concerns about changing HostResolveImportedModule, but as I outlined in the OP, I don't think the changes are really that invasive, and would not affect the module loading path at all. All existing parts of the pipeline would not call it with a first argument that's a script, and any environment which wants to not support scripts with this feature would just need to add some kind of "If currentScriptOrModule is a Script Record, throw a TypeError" line to their HostResolveImportedModule. I'm happy to dive in to any mor edetailed concerns you might have.

I don't understand await import that well, but my understanding is that this would be complementary, for the truly-dynamic cases where you want a promise and all the flexibility it brings. For example, Promise.all or Promise.raceing several loads in parallel, or passing to a web platform API that accepts a promise like event.respondWith. I am also not sure whether await import allows arbitrary strings (like import(./languagepack/${navigator.language}.js)). await import is for the more static case, from what I understand, allowing it to be integrated into tree-shaking tools and so on.

[domenic]

I've put together a proposal at https://github.com/domenic/proposal-import-function where the readme explains some of the things we discussed, @caridy. The spec is relatively complete, too: https://domenic.github.io/proposal-import-function/. Looking forward to working with you all on this!