-
-
Notifications
You must be signed in to change notification settings - Fork 252
Upgrade to TypeScript v5.0 and set module{,Resolution} option to Node16
#3645
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
f14624e to
0029d3b
Compare
|
I worked on this a bit and by bumping some packages and also using a local copy of I haven't tried running |
ee34373 to
5b2d07d
Compare
3ee4229 to
26937bc
Compare
module, moduleResolution options to Node16module, moduleResolution options to NodeNext
49aa7ea to
92db09e
Compare
92db09e to
94b220f
Compare
module, moduleResolution options to NodeNext^5.0.4 and set module, moduleResolution to NodeNext
f5d774c to
e0487db
Compare
b459ff2 to
5cc2651
Compare
|
👍 Dependency issues cleared. Learn more about Socket for GitHub ↗︎ This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored. Ignoring: Next stepsTake a deeper look at the dependencyTake a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support [AT] socket [DOT] dev. Remove the packageIf you happen to install a dependency that Socket reports as Known Malware you should immediately remove it and select a different dependency. For other alert types, you may may wish to investigate alternative packages or consider if there are other ways to mitigate the specific risk posed by the dependency. Mark a package as acceptable riskTo ignore an alert, reply with a comment starting with |
0454e0e to
903468e
Compare
…erstruct` in all core nested dependencies
c4cc6dd to
9ebfa84
Compare
…7.8.0`, `@metamask/snaps-controllers` to `^9.3.0`
9ebfa84 to
59fe394
Compare
c273024 to
6b0075d
Compare
…mask/providers` from `17.0.0` to `17.1.1`
986f16f to
f50b209
Compare
MajorLift
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Notes for reviewers
| const cid = index !== -1 ? url.substring(0, index) : url; | ||
| const path = index !== -1 ? url.substring(index) : undefined; | ||
|
|
||
| const { CID } = await import('multiformats'); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
multiformats is an ESM-only package, and we are replacing its import statement with dynamic import syntax.
This causes the following changes:
getIpfsCIDv1AndPath,getFormattedIpfsUrlare now async functions (this is a breaking change forassets-controllers).- The
--experimental-vm-modulesflag needs to be prepended to our build scripts that runjest, with accompanying updates to our yarn constraints.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How did this work previously if it's ESM-only?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So far as I can tell, our previous moduleResolution option node/node10 will happily pass ESM-only packages into require calls without raising any warnings.
So it's likely the warnings we're now seeing with node16 should have been there all along. The package migrated to ESM 4 years ago (multiformats/js-multiformats@f925195#diff-7ae45ad102eab3b6d7e7896acd08c427a9b25b346470d7bc6507b6481575d519).
At runtime, either the module was interoperable by coincidence (not quite sure if this is possible or likely -- maybe if there's no ESM-specific syntax?), or it was breaking and we didn't notice.
|
|
||
| % All non-root packages must have the same "test" script. | ||
| gen_enforced_field(WorkspaceCwd, 'scripts.test', 'jest --reporters=jest-silent-reporter') :- | ||
| gen_enforced_field(WorkspaceCwd, 'scripts.test', 'NODE_OPTIONS=--experimental-vm-modules jest --reporters=jest-silent-reporter') :- |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The --experimental-vm-modules flag is necessary if there are dynamic imports in the compiled code used by our tests.
| export type { AutoManagedNetworkClient } from './create-auto-managed-network-client'; | ||
| export * from './NetworkController'; | ||
| export * from './constants'; | ||
| export type { BlockTracker, Provider } from './types'; | ||
| export type { NetworkClientConfiguration } from './types'; | ||
| export type { | ||
| NetworkClientConfiguration, | ||
| InfuraNetworkClientConfiguration, | ||
| CustomNetworkClientConfiguration, | ||
| } from './types'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These exports accommodate the new import statements in Token{Detection,Rate}Controller.test.ts.
Previously, these types were accessed through subpath imports, which are now not supported unless specified in the library's package.json exports field.
| - '@metamask/create-release-branch' | ||
| - 'depcheck' | ||
| - 'eslint-interactive' | ||
| - 'rimraf' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only listed as a devDep in the root package.json, where it's used in a build script but not in any modules.
| import { useFakeTimers } from 'sinon'; | ||
|
|
||
| import { advanceTime } from '../../../tests/helpers'; | ||
| import { createMockInternalAccount } from '../../accounts-controller/src/tests/mocks'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With the Node{16,Next} option, subpath imports are no longer supported unless explicitly defined in the source package manifest's "exports" field.
Here, an import from @metamask/accounts-controller/src/tests/mocks needed to be replaced with a relative path import.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah I'm not sure how this slipped through the cracks! Makes sense.
| "pre-push": "yarn lint" | ||
| }, | ||
| "resolutions": { | ||
| "@metamask/snaps-sdk@6.1.0/@metamask/providers": "17.1.1", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Fixes build error due to package version mismatch: MetaMask/core/actions/runs/10011688901/job/27675682526?pr=3645
- Removal is blocked by a
@metamask/snaps-sdkrelease with@metamask/providersbumped to>=17.1.1.
| ### Uncategorized | ||
|
|
||
| - Please hold off on new releases of this package until the yarn resolution for `@metamask/providers` is removed. | ||
| - This is blocked by a `@metamask/snaps-sdk` release with `@metamask/providers` bumped to `>=17.1.1`. | ||
| - See: [Fix regressions introduced by @metamask/providers@17.1.1](https://github.com/MetaMask/snaps/pull/2579) | ||
| - Build error fixed by yarn resolution: [MetaMask/core/actions/runs/10011688901/job/27675682526?pr=3645](https://github.com/MetaMask/core/actions/runs/10011688901/job/27675682526?pr=3645) | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The packages affected by the yarn resolution are {accounts,chain,profile-sync}-controller. It's safe to temporarily merge the yarn resolution into main, as long as these packages aren't released (and even if they are @metamask/providers is not a direct dependency of any of these packages).
In any case, these three packages will be excluded from the core releases that follow this PR, since they are blocked by a snaps-controllers release as well.
mcmire
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| "@metamask/eth-json-rpc-provider": "^4.1.1", | ||
| "@metamask/ethjs-provider-http": "^0.3.0", | ||
| "@metamask/keyring-api": "^8.0.0", | ||
| "@metamask/keyring-api": "^8.0.1", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the @MetaMask/confirmations team: this is the only change in this PR to a confirmations-owned package (aside from the typescript bumps and test build script updates on every package).
…tion for `@metamask/providers` (#4547) ## References - Contributes to #3651 - Follows from #3645 ## Changelog ### `@metamask/accounts-controller` ```md ### Changed - Bump `@metamask/snaps-sdk` from `^6.1.0` to `^6.1.1` ([#4547](#4547)) - Bump `@metamask/snaps-utils` from `^7.8.0` to `^7.8.1` ([#4547](#4547)) ``` ### `@metamask/chain-controller` ```md ### Changed - Bump `@metamask/snaps-controllers` from `^9.3.0` to `^9.3.1` ([#4547](#4547)) - Bump `@metamask/snaps-sdk` from `^6.1.0` to `^6.1.1` ([#4547](#4547)) - Bump `@metamask/snaps-utils` from `^7.8.0` to `^7.8.1` ([#4547](#4547)) ``` ### `@metamask/profile-sync-controller` ```md ### Changed - Bump `@metamask/snaps-controllers` from `^9.3.0` to `^9.3.1` ([#4547](#4547)) - Bump `@metamask/snaps-sdk` from `^6.1.0` to `^6.1.1` ([#4547](#4547)) - Bump `@metamask/snaps-utils` from `^7.8.0` to `^7.8.1` ([#4547](#4547)) ``` ## Checklist - [x] I've updated the test suite for new or updated code as appropriate - [x] I've updated documentation (JSDoc, Markdown, etc.) for new or updated code as appropriate - [x] I've highlighted breaking changes using the "BREAKING" category above as appropriate
## Explanation Update the template used by the `create-package` script. The previous template was no longer compatible with our Yarn constraints. ## References Here are the PRs related to these specific changes: * #4648 * #3645 * #1390 * #3668 ## Changelog N/A ## Checklist - [x] I've updated the test suite for new or updated code as appropriate - [x] I've updated documentation (JSDoc, Markdown, etc.) for new or updated code as appropriate - [x] I've communicated my changes to consumers by [updating changelogs for packages I've changed](https://github.com/MetaMask/core/tree/main/docs/contributing.md#updating-changelogs), highlighting breaking changes as necessary - [x] I've prepared draft pull requests for clients and consumer packages to resolve any breaking changes

Explanation
TypeScript v5.0
As part of the Wallet Framework Team's OKRs (Q2 2024 O3KR4, Q3 2024 O2KR4), we are upgrading TypeScript to v5.0+ for all packages in the core monorepo.
These upgrades will give us access to new features, aid us in writing more type-safe and modern code, and also allow us to reach parity with Extension and other MetaMask projects.
Node16In order to maximize the benefits of this upgrade, we are also enabling the
Node16setting for themoduleandmoduleResolutiontsconfig options.Motivation
Interop: CJS modules referencing ESM modules
The core monorepo is a collection of CJS packages, which use CJS module resolution rules internally, and are treated as CJS modules by Node.js. This is true despite that fact that these packages are authored in TypeScript using ESM syntax (
import/exportstatements) and are set up to output dual builds for both CJS and ESM.With
Node16orNodeNextenabled, CJS modules are unable to reference ESM modules via static/synchronousimportstatements, as TypeScript assumes them to be compiled down to CJS-onlyrequirestatements.There are three solutions for this issue, of which we are utilizing the first two:
importstatements with dynamic import syntax (which, based on CJS emit rules, are not transformed torequirestatements)."type": "module"in package.json and renaming all of our scripts to *.cjs)With dependencies that we control or use extensively, we pursue the first option, as we're doing with
superstructand@metamask/utils(and all of the many core dependencies that are downstream of either or both).With dependencies that see more limited usage, we are opting for either the second option (e.g.
multiformats) or, if available, using a CJS-compatible alternative (e.g. replacinglodash-eswithlodash).The third solution of migrating to ESM is the most fundamental, long-term measure, but we are refraining from it at this stage until we can make a concerted effort to migrate our codebase as a whole. This is because any individual ESM migration can cause a cascading effect through the dependency tree where other packages are now required to migrate as well.
Reasons for moving away from
node/node10The following outlines the motivation for switching to
Node16, backed by relevant entries from the official TypeScript documentation.moduleResolutionsettingnodeis renamed tonode10, and strongly discouraged from usage.node16andnodenextare the only module options that reflect the complexities of Node.js’s dual module system, they are the only correct module options for all apps and libraries that are intended to run in Node.js v12 or later, whether they use ES modules or not."node10setting is unable to guarantee correct module resolution for ESM-only dependencies.node16andnodenextdescribe the full range of behavior for Node.js’s dual-format module system, and emit files in either CommonJS or ESM format. This is different from every othermoduleoption, which are runtime-agnostic and force all output files into a single format, leaving it to the user to ensure the output is valid for their runtime."node10setting does not support the package.json"exports"field, which is used in our libraries to expose dual builds and type declarations.Node16andNodeNextsettings maximize downstream compatibility."moduleResolution": "nodenext"is only checking that the output works in Node.js, but in most cases, module code that works in Node.js will work in other runtimes and in bundlers"Node16vs.NodeNextThe two settings are currently identical, and
Node16is intended to work with all current node versions v16 or higher. If additional capabilities are added toNodeNextorNode18/Node20that we want to apply to our codebases, we will be able to introduce them after checking for disruptive regressions or breaking changes.Relationship with other tsconfig options
--modulenodenextornode16implies and enforces themoduleResolutionwith the same name (and vice versa).--modulenode16implies (up to)--targetes2022.--modulenodenextimplies (up to)--targetesnext.Description
superstructdependency with@metamask/superstruct^3.0.0.^3.1.0superstructimport statements with@metamask/superstruct@metamask/utilsto^8.5.0.^9.0.0@metamask/superstruct@npm:3.1.0typescriptto~5.0.4moduleandmoduleResolutiontsconfig options toNode16Bump TypeScript module output optionstargetandlibtoES2022#4507Further context on why the
superstructandutilschanges are necessary:tsupfor bundling utils#144ts-bridgeas build tool, setmoduleResolutiontoNodeNextsuperstruct#18ts-bridgefor building utils#182Release order roadmap
Due to interdependencies between the packages involved in this PR, we will need to update and release them in a specific order:
@metamask/utilsto^9.0.0,@metamask/rpc-errorsto^6.3.1#4516{base,permission}-controllersnaps-sdk,snaps-utils,snaps-controllers,keyring-api)superstructwith ESM-compatible fork@metamask/superstructsnaps#2445snaps-sdk,snaps-utils,keyring-apimodule{,Resolution}option toNode16#3645@metamask/providersvia@metamask/snaps-sdkto17.1.1{accounts,chain,profile-sync}-controller) to hold off on new releasessnaps-controllersand consumers ofutils)snaps-controllersas dependency, and are affected by@metamask/providersyarn resolution{accounts,chain,profile-sync}-controllersnaps-controllers(Release 58.0.0 snaps#2595){accounts,chain,profile-sync}-controller,eth-snap-keyring(chore(deps): replacesuperstructimports with@metamask/superstructeth-snap-keyring#311)@metamask/snaps-{controllers,sdk,utils}and remove yarn resolution for@metamask/providers#4547snapspackages while releasing@metamask/utils@9.0.0version bumps for all dependencies and nested dependencies)References
superstructts-bridgeas build tool, setmoduleResolutiontoNodeNextsuperstruct#18utilssubmodule as package-level exports superstruct#25utils@metamask/superstruct, setmoduleResolutiontoNodeNextutils#185@metamask/superstructfrom^3.0.0to^3.1.0utils#194superstruct,utils:abi-utils:@metamask/superstructto^3.1.0,@metamask/utilsto^9.0.0abi-utils#80chain-api:superstructwith@metamask/superstruct+ bump@metamask/utilsaccounts-chain-api#5eth-simple-keyring:@metamask/utilsfrom^8.1.0to^9.0.0eth-simple-keyring#177providers:tsupwithts-bridgeas build tool providers#336@metamask/utilsto^9.0.0, bump@metamask/{json-rpc-engine,json-rpc-middleware-stream,rpc-errors}providers#345rpc-errors:@metamask/utilsfrom^8.3.0to^9.0.0rpc-errors#147snaps-registry:@metamask/superstructto^3.1.0,@metamask/utilsto^9.0.0snaps-registry#693snapsmonorepo releasessnaps-sdk,snaps-utilskeyring-apisuperstructimports with@metamask/superstructkeyring-api#328snaps-controllerseth-snap-keyringsuperstructimports with@metamask/superstructeth-snap-keyring#311Changelog
@metamask/accounts-controller@metamask/assets-controllers(major)@metamask/chain-controller@metamask/keyring-controller@metamask/network-controller(minor)@metamask/profile-sync-controller@metamask/transaction-controller@metamask/user-operation-controllerChecklist