await import() doesn't work in REPL even with --experimental-repl-await

[fatcerberus]

• Version: v10.0.0
• Platform: Windows 10 Pro 18.03 x64
• Subsystem: REPL

await import() doesn't work in the REPL even with --experimental-repl-await:

C:\Users\fatce>node --experimental-modules --experimental-repl-await
> (node:4220) ExperimentalWarning: The ESM module loader is experimental.
> await Promise.resolve(812)
812
> await import("module.mjs")
await import("module.mjs")
^^^^^

SyntaxError: await is only valid in async function

I can verify that dynamic import by itself works in the REPL, it's only the await that's an issue:

> import("module.mjs")
Promise {
  <pending>,
  domain:
   Domain {
     domain: null,
     _events:
      { removeListener: [Function: updateExceptionCapture],
        newListener: [Function: updateExceptionCapture],
        error: [Function: debugDomainError] },
     _eventsCount: 3,
     _maxListeners: undefined,
     members: [] } }
> (node:4220) UnhandledPromiseRejectionWarning: TypeError [ERR_INVALID_URL]: Invalid URL: repl
    at onParseError (internal/url.js:234:17)
    at parse (internal/url.js:243:3)
    at new URL (internal/url.js:318:5)
    at normalizeReferrerURL (internal/process/esm_loader.js:22:10)
    at setImportModuleDynamicallyCallback (internal/process/esm_loader.js:66:37)
    at process._tickCallback (internal/process/next_tick.js:178:7)
(node:4220) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:4220) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
\>

[guybedford]

Interestingly Chrome itself has the exact same issue - https://bugs.chromium.org/p/chromium/issues/detail?id=807528.

[devsnek]

it's worth noting that currently import() doesn't work in repl anyway. there is a pr open to fix that but it hasn't landed yet.

[jdalton]

As you all iterate on it from the parse side I have a pretty good handle on acorn and the ins-and-outs of making it to work (up to the latest --harmony syntax supported in Node 10).

Feel free to cc me in to update/tweak/improve bits around it.

[guybedford]

@jdalton could you share the link to your branch / work here further?

[jdalton]

@guybedford Sure! For the parse-side (so non-dynamic-import base url issues) I have acorn extensions and customizations handled in a repo (here - MIT licensed). When compiled into Node I update the Acorn parser references to improve parsing. There is still some tweaks to do to the AST transforms done by the --experimental-repl-await but the syntax parsing itself can be improved at least. One of the things I leverage is the fact that Acorn doesn't need to perform runtime validation checks (since v8/chakra will handle those) this means parsing can be a bit more loose.

There is still some tweaks to do to the AST transforms done by the --experimental-repl-await

By this I mean updating the transform to handle statements a bit better.
For example, assuming static import worked in the REPL (I know, just roll with it), the following...

> import "path";var { sep } = await import("path")
SyntaxError: await is only valid in async function

would error because the transform will fail (falling back to the untransformed literal code).
It fails because the transform currently results in something like

(async () => { import "path";void ({ sep } = await import("path")) })()

and static import inside a function is an error.

The improved parsing bit is just adding Acorn parsing support for things like dynamic import, import.meta, class fields, etc. which is what these are for.

I also have a toolbox of fast-paths/optimization techniques for Acorn to make things snappy. This is all a balance between how much tinkering with Acorn is wanted vs. waiting on a standardized top-level await. For parsing support at least updating and optimization Acorn is straight forward enough.

[guybedford]

@jdalton thanks, that helps a lot to get context into the issue here.

[rubys]

Duplicate of #19570 ?

[fatcerberus]

No, #19570 is about import() itself not functioning properly under repl (it always fails and returns a rejected promise). This issue is a parsing bug - await followed by import() incorrectly produces a syntax error.

[TimothyGu]

Ah, that's because Acorn doesn't implement dynamic import() yet, since the proposal is stuck at Stage 3 in the TC39 process. Not much we could do about it, though https://github.com/kesne/acorn-dynamic-import might help?

[jdalton]

@TimothyGu I covered this in the comment above.

[Jamesernator]

This seems to be working now in Node 13 with node --experimental-repl-await --experimental-modules.