Flaky es-module/test-esm-loader-side-effect (and other ESM loader tests)

[tniessen]

Test

es-module/test-esm-loader-side-effect

Platform

FreeBSD, Linux x64, SmartOS, Windows

Console output

00:28:00 not ok 3550 es-module/test-esm-loader-side-effect
00:28:00   ---
00:28:00   duration_ms: 120140.19600
00:28:00   severity: fail
00:28:00   exitcode: -15
00:28:00   stack: |-
00:28:00     timeout
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00     (node:46152) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
00:28:00     (Use `node --trace-warnings ...` to show where the warning was created)
00:28:00   ...
00:28:00 not ok 3551 es-module/test-esm-loader-spawn-promisified
00:28:00   ---
00:28:00   duration_ms: 120131.32000
00:28:00   severity: fail
00:28:00   exitcode: -15
00:28:00   stack: |-
00:28:00     timeout
00:28:00     TAP version 13
00:28:00     # Subtest: Loader hooks throwing errors
00:28:00         # Subtest: throws on nonexistent modules
00:28:00         ok 1 - throws on nonexistent modules
00:28:00           ---
00:28:00           duration_ms: 2176.040127
00:28:00           ...
00:28:00         # Subtest: throws on unknown extensions
00:28:00         ok 2 - throws on unknown extensions
00:28:00           ---
00:28:00           duration_ms: 13980.00791
00:28:00           ...
00:28:00         # Subtest: throws on invalid return values
00:28:00         ok 3 - throws on invalid return values
00:28:00           ---
00:28:00           duration_ms: 4010.533867
00:28:00           ...
00:28:00         # Subtest: throws on boolean false
00:28:00         ok 4 - throws on boolean false
00:28:00           ---
00:28:00           duration_ms: 37799.667001
00:28:00           ...
00:28:00         # Subtest: throws on boolean true
00:28:00         ok 5 - throws on boolean true
00:28:00           ---
00:28:00           duration_ms: 17616.354417
00:28:00           ...
00:28:00         # Subtest: throws on invalid returned object
00:28:00         ok 6 - throws on invalid returned object
00:28:00           ---
00:28:00           duration_ms: 37620.543006
00:28:00           ...
00:28:01         # Subtest: throws on unsupported format
00:28:01         ok 7 - throws on unsupported format
00:28:01           ---
00:28:01           duration_ms: 25106.204444
00:28:01           ...
00:28:01         # Subtest: throws on invalid format property type
00:28:01         ok 8 - throws on invalid format property type
00:28:01           ---
00:28:01           duration_ms: 24915.760456
00:28:01           ...
00:28:01         # Subtest: rejects dynamic imports for all of the error cases checked above
00:28:01         ok 9 - rejects dynamic imports for all of the error cases checked above
00:28:01           ---
00:28:01           duration_ms: 64962.084782
00:28:01           ...
00:28:01         1..9
00:28:01     ok 1 - Loader hooks throwing errors
00:28:01       ---
00:28:01       duration_ms: 65469.146812
00:28:01       type: 'suite'
00:28:01       ...
00:28:01     # Subtest: Loader hooks parsing modules
00:28:01         # Subtest: can parse .js files as ESM
00:28:01         ok 1 - can parse .js files as ESM
00:28:01           ---
00:28:01           duration_ms: 17015.979174
00:28:01           ...
00:28:01         # Subtest: can define .ext files as ESM
00:28:01         ok 2 - can define .ext files as ESM
00:28:01           ---
00:28:01           duration_ms: 23380.46326
00:28:01           ...
00:28:01   ...

Build links

Many. See nodejs/reliability#543, for example.

Additional information

Refs: nodejs/reliability#543

cc @JakobJingleheimer, perhaps this could be related to #44710?

[merceyz]

Possibly the same issue as #47566

[GeoffreyBooth]

@nodejs/loaders

[JakobJingleheimer]

Yes, I think so. Antoine noted that these are all related to async messaging (warnings etc), which are known to be flaky after #44710.

We're going to remove the deprecation warnings (and their tests) for old hooks as they were deprecated multiple major versions ago (I wanna say v16 line); that will eliminate some of the flakiness.

We'll need to add specific handling for anything else that needs to communicate/coordinate with the main thread (such as writing to stdout).

[GeoffreyBooth]

Is this still an issue after the recent PRs that have landed?

[aduh95]

According to nodejs/reliability#545, no flaky ESM test have been reported since 3 days ago 🥳