feat: add support for ESM presets

[sheerlox]

This PR adds support for loading pure ESM presets. This was not previously possible because import-from uses createRequire under the hood.

In order to replace it, I've created an ESM module loader that abstracts the preset loading logic for semantic-release repos (first tries to hit node_modules, then if not found tries to hit the relative path from the current working directory).

I've tested these changes locally on the https://github.com/insurgent-lab/conventional-changelog-preset repo (which is a preset but also uses its current version for releasing with semantic-release), both with the current latest version and the ESM version (which you can find in the refactor/esm branch since it's probably the only conventional-changelog ESM preset at this time).

Related

• feat: add support for ESM presets release-notes-generator#544
• ES Modules semantic-release#2133
• ES Module release-notes-generator#296
• discussion on refactor(esm): converted the package to esm #297 (comment)

cc @travi 😉

[sheerlox]

See semantic-release/release-notes-generator#544 (comment).

[sheerlox]

Looks like the tests are broken on Windows. I'll investigate and come back to you when I have fixed the issue.

Thanks for the follow-up and enthusiasm on this! 🎉

[sheerlox]

The dynamic import on Windows requires absolute paths to be an URL. This has been fixed in v1.0.3.

Both PRs have been rebased against master and upgraded to use import-from-esm@1.0.3 🎉

[travi]

thanks a bunch for working through these and for your patience for me to get a chance to review

[sheerlox]

happy to contribute!

[github-actions]

🎉 This PR is included in version 11.1.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[travi]

happy to contribute!

if you'd like to help us out on a more official basis, i'd be happy to invite you to the org and grant you triage rights to start with. let me know if that is something you'd be interested in, but no pressure if you arent interested

[sheerlox]

if you'd like to help us out on a more official basis, i'd be happy to invite you to the org and grant you triage rights to start with. let me know if that is something you'd be interested in, but no pressure if you arent interested

Sure, I'll give it my best shot, thank you for the invite!
Note that I'm currently evolving toward Elixir & BEAM (for which I'm working on a semantic-release plugin 👀), but I'll keep maintaining TS/JS packages I'm involved with, and would greatly appreciate contributing more to semantic-release on my (rarifying) free time 😄

[mcmiddle592]

@sheerlox @travi I just pulled this change into my project and am facing an issue in my Windows Jenkins environment while running "npm run semantic-release":

[semantic-release] An error occurred while running semantic-release: Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
at new NodeError (node:internal/errors:405:5)
at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:136:11)
at defaultLoad (node:internal/modules/esm/load:87:3)
at nextLoad (node:internal/modules/esm/loader:163:28)
at ESMLoader.load (node:internal/modules/esm/loader:603:26)
at ESMLoader.moduleProvider (node:internal/modules/esm/loader:457:22)
at new ModuleJob (node:internal/modules/esm/module_job:64:26)
at #createModuleJob (node:internal/modules/esm/loader:480:17)
at ESMLoader.getModuleJob (node:internal/modules/esm/loader:434:34) {
code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}
Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
at new NodeError (node:internal/errors:405:5)
at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:136:11)
at defaultLoad (node:internal/modules/esm/load:87:3)
at nextLoad (node:internal/modules/esm/loader:163:28)
at ESMLoader.load (node:internal/modules/esm/loader:603:26)
at ESMLoader.moduleProvider (node:internal/modules/esm/loader:457:22)
at new ModuleJob (node:internal/modules/esm/module_job:64:26)
at #createModuleJob (node:internal/modules/esm/loader:480:17)
at ESMLoader.getModuleJob (node:internal/modules/esm/loader:434:34) {
code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}04:49:55.031000 common.go:40: exit status 1

I wasn't expecting a breaking change, is there something that needs to be done in my project to handle this?

[sheerlox]

Hello @mcmiddle592, I'm sorry you're encountering an error because of that PR.

Could you please open an issue and share more details about the configuration you're using so I can better understand where the issue is coming from?

Note: I would expect import-from-esm to be mentioned in the stack trace, but that's not the case. Maybe it just isn't long enough.

[sheerlox]

That's definitely an issue with import-from-esm not covering the case where a Windows user could pass an absolute path. Please open an issue there instead to track this @mcmiddle592, but please know I'm already working on a fix.

[sheerlox]

@mcmiddle592 the fix has been released, please run npm update import-from-esm to install the fix and update your package-lock.json. My apologies for the inconvenience.

[travi]

Since semantic-release/semantic-release#3037 was also released on Friday, I'm curious if that change is involved as well.

[sheerlox]

Yeah, that's definitely possible since resolve-from does not return an URL.
One workaround could be adding JSON support in import-from-esm so it could be used directly in place of resolve-from.

[dominykas]

Yes, it could be related, sorry - I do not have Windows to test this myself either.

Would the fix to prepend file:// be enough for this?

Then again - the only code paths that were changed were when the -e option is used - which is not the case here in the scripts? Would it make sense to pin to an earlier version of semantic-release (v22.0.6, I guess) to double check?

[sheerlox]

Would the fix to prepend file:// be enough for this?

That should solve the issue but feels a bit hackier than directly using import-from-esm, which would have the benefits of delegating resolution responsibility to it and unifying how we load configuration files (whether presets or extends). I'm planning to release JSON support in a few hours.

Then again - the only code paths that were changed were when the -e option is used - which is not the case here in the scripts? Would it make sense to pin to an earlier version of semantic-release (v22.0.6, I guess) to double check?

I'm not sure I understand what you mean, but the issue seems to arise because the user specifies an absolute path in its "extends" configuration/CLI option.

EDIT: JSON modules support released in v1.2.0

[mcmiddle592]

@sheerlox @travi thanks for all the help here and the quick responses. I pinned my Semantic-Release in the short term to 22.0.6 to workaround this issue. I won't be able to get to testing this until next week, but when I do I plan to ensure I am using the latest version of import-from-esm (instead of import-from) and see if my issue on Windows is resolved. If there is anything else I need to check then let me know.

[dominykas]

Sorry, I might not be following things in full - I responded because there was a change in semantic-release due to how it's loading the --extend configs, but it seems there was a similar change in @semantic-release/commit-analyzer, and the issue here is contained inside of @semantic-release/commit-analyzer? Or is there an actual problem with semantic-release itself? Or possibly both of course...

[sheerlox]

We think it might be both yes. As of now, the issue in @semantic-release/commit-analyzer has been fixed in release 1.1.3 of import-from-esm.

Since I added JSON module support yesterday, I just tried fixing the issue in semantic-release by using that library, which works well overall, except there's a test that tries to import a node_modules package containing a single index.json file (and no package.json). Currently, the library isn't able to handle that (and it is not specified in the NodeJS docs, although require seems to handle it correctly), but I think it wouldn't even work with an export field pointing to that file in package.json.

Unfortunately, I won't be available in the next few days so I won't be able to work on that issue. Maybe it would be worth prepending file:// in the meantime until I can tackle that issue in import-from-esm?

[travi]

sorry for the slow response from me. the last couple days have been pretty busy for me. thank you for keeping the conversation moving forward.

since it sounds like any issue here is likely resolved, lets move the conversation about extending a config back to the other issue.

@mcmiddle592 we appreciate you remaining engaged and are interested in the results of your test with the import-from-esm update. as you can tell from the conversation, we suspect that there is also an update needed in core semantic-release for the other recent change related to esm loading. still interested in your results, but don't be surprised if you still encounter a problem until we make the other update.

[seebeen]

Yes, it could be related, sorry - I do not have Windows to test this myself either.

@dominykas I'm on Windows. What needs doing? :)

[mcmiddle592]

@sheerlox @travi @dominykas just wanted to confirm that I did test replacing "import-from" with "import-from-esm" in my projects package-lock.json (by reinstalling my semantic release dependencies). This however did not resolve the issue (which I believe was expected), I still have the same problem:

[semantic-release] An error occurred while running semantic-release: Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
at new NodeError (node:internal/errors:405:5)
at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:136:11)
at defaultLoad (node:internal/modules/esm/load:87:3)
at nextLoad (node:internal/modules/esm/loader:163:28)
at ESMLoader.load (node:internal/modules/esm/loader:603:26)
at ESMLoader.moduleProvider (node:internal/modules/esm/loader:457:22)
at new ModuleJob (node:internal/modules/esm/module_job:64:26)
at #createModuleJob (node:internal/modules/esm/loader:480:17)
at ESMLoader.getModuleJob (node:internal/modules/esm/loader:434:34) {
code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}
Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
at new NodeError (node:internal/errors:405:5)
at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:136:11)
at defaultLoad (node:internal/modules/esm/load:87:3)
at nextLoad (node:internal/modules/esm/loader:163:28)
at ESMLoader.load (node:internal/modules/esm/loader:603:26)
at ESMLoader.moduleProvider (node:internal/modules/esm/loader:457:22)
at new ModuleJob (node:internal/modules/esm/module_job:64:26)
at #createModuleJob (node:internal/modules/esm/loader:480:17)
at ESMLoader.getModuleJob (node:internal/modules/esm/loader:434:34) {
code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'
}10:48:07.392620 common.go:40: exit status 1

[sheerlox]

Thanks for confirming the issue @mcmiddle592, we're now tracking this in semantic-release/semantic-release#3037. We'll let you know when the issue is resolved.

[kerasing]

@travi - hey this was a breaking change for me, because I was using require to access analyzeCommits to get the release type outside of semantic-release (in a github action i use to pre-emptively ensure the maintenance branches are within their range boundaries), and this shifted to requiring import.

[sheerlox]

hey this was a breaking change for me, because I was using require to access analyzeCommits to get the release type outside of semantic-release (in a github action i use to pre-emptively ensure the maintenance branches are within their range boundaries), and this shifted to requiring import.

Hi @kerasing, as you can see on this line of the PR diff, the function signature of analyzeCommits hasn't changed. Also, this package was already an ESM module before this PR.

The package was converted to ESM in v10 on June 2nd, 2023.

Is there something I'm missing about my PR changes that caused you to run into an issue?