[Bug]: Yarn 3 support?

[azat-io]

Which Remix packages are impacted?

• remix (Remix core)
• create-remix

What version of Remix are you using?

1.0.5

Steps to Reproduce

Trying to install dependencies with Yarn berry. My current Yarn version is v3.2.0-rc.5.

Log file:

# This file contains the result of Yarn building a package (azat-io@workspace:.)
# Script name: postinstall

(node:67592) ExperimentalWarning: --experimental-loader is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
EROFS: read-only filesystem, open '/node_modules/remix/package.json'

Dependencies in my package.json:

{
  "dependencies": {
    "@remix-run/node": "1.0.5",
    "@remix-run/react": "1.0.5",
    "@remix-run/serve": "1.0.5",
    "@remix-run/server-runtime": "1.0.5",
    "react": "17.0.2",
    "react-dom": "17.0.2",
    "remix": "1.0.5"
  },
  "devDependencies": {
    "@remix-run/dev": "1.0.5",
    "@types/react": "17.0.24",
    "@types/react-dom": "17.0.9",
    "typescript": "4.1.2"
  }
}

Expected Behavior

Installation successful

Actual Behavior

Installation failed

[mxro]

I am able to install on yarn v 3.0.2 but when running yarn build using remix@1.0.5 I get the following error:

Building Remix app in production mode...

 > error: [plugin: remix-mdx] Cannot find package 'xdm' imported from C:\Users\Max\repos\goldstack-mega\.yarn\cache\@remix-run-dev-npm-1.0.6-73f0b55172-2fca586246.zip\node_modules\@remix-run\dev\compiler\plugins\mdx.js
Did you mean to import xdm-npm-2.1.0-e825e62593-5a3f2434a5.zip/node_modules/xdm/index.js?


Build failed with 1 error:
error: Cannot find package 'xdm' imported from C:\Users\Max\repos\goldstack-mega\.yarn\cache\@remix-run-dev-npm-1.0.6-73f0b55172-2fca586246.zip\node_modules\@remix-run\dev\compiler\plugins\mdx.js
Did you mean to import xdm-npm-2.1.0-e825e62593-5a3f2434a5.zip/node_modules/xdm/index.js?
Built in 170ms

Assuming this may also be related to module resolution using yarn 2?

[jacob-ebey]

What node version are you on?

[mxro]

I'm using node v14.17.6

[mxro]

Also getting the following error when trying to run yarn postinstall:

$ yarn postinstall
Missing the "remix" package. Please run `npm install remix` before `remix setup`.
Successfully setup Remix for node.

But definitely did an install before running this!

[jacob-ebey]

Seems like a linking issue. I don't have time to play with Yarn3 right now but if you would like to investigate and report back that would be awesome. We generally recommend to use NPM as that's what we test against, but we also use yarn1 without issue.

[mxro]

Okay, thanks for having had a quick look into it and understand if this cannot be looked at further. Wouldn't really know where to start, but will check in again in a few months if support for Yarn 2/3/'Berry' would be provided by then.

[rhefner]

Just an FYI - I got remix to work a while back (as in March/April) with Yarn v2 PnP by patching @remix-run/dev:

diff --git a/compiler/assets.js b/compiler/assets.js
index c161dd31793837b2ce722ddbaa496f3d9c6b2bac..ef5989a66f30941bdb9cc884e70f62ec346c231b 100644
--- a/compiler/assets.js
+++ b/compiler/assets.js
@@ -30,7 +30,7 @@ async function createAssetsManifest(config, metafile) {
   for (let key of Object.keys(metafile.outputs).sort()) {
     let output = metafile.outputs[key];
     if (!output.entryPoint) continue;
-    let entryPointFile = path.resolve(output.entryPoint.replace(/(^browser-route-module:|\?browser$)/g, ""));
+    let entryPointFile = path.resolve(output.entryPoint.replace(/(^pnp:|^browser-route-module:|\?browser$)/g, ""));
 
     if (entryPointFile === entryClientFile) {
       entry = {
diff --git a/compiler.js b/compiler.js
index 86e2c589195f8e27560696891c09583d32d09e63..e79e849cd877ab10ced733f01a517bdf14aaabb9 100644
--- a/compiler.js
+++ b/compiler.js
@@ -9,6 +9,7 @@ var module$1 = require('module');
 var esbuild = require('esbuild');
 var debounce = require('lodash.debounce');
 var chokidar = require('chokidar');
+var { pnpPlugin } = require('@yarnpkg/esbuild-plugin-pnp')
 var build$1 = require('./build.js');
 var config = require('./config.js');
 var invariant = require('./invariant.js');
@@ -194,7 +195,7 @@ async function createBrowserBuild(config, options) {
     define: {
       "process.env.NODE_ENV": JSON.stringify(options.mode)
     },
-    plugins: [browserRouteModulesPlugin(config, /\?browser$/), emptyModulesPlugin(config, /\.server(\.[jt]sx?)?$/)]
+    plugins: [browserRouteModulesPlugin(config, /\?browser$/), emptyModulesPlugin(config, /\.server(\.[jt]sx?)?$/), pnpPlugin()]
   });
 }
 
@@ -234,7 +235,7 @@ async function createServerBuild(config, options) {
       }
 
       return false;
-    })]
+    }), pnpPlugin()]
   });
 }

The patch above of course won't work with the latest code, but back then it was possible to get it to work with PnP.. Remix does work with Yarn v3 if you're using nodeLinker: node-modules - If you're in a monorepo, you'll likely need to add the following to the Remix packages' package.json as well:

"installConfig": {
  "hoistingLimits": "workspaces"
}

b/c Remix gets hoisted to the top-level node_modules/:

Watching Remix app in development mode...

 > ../../node_modules/remix/browser/index.js:11:14: error: Could not resolve "./client"
    11 │ export * from './client';
       ╵               ~~~~~~~~~~

  > ../../node_modules/remix/browser/index.js:12:14: error: Could not resolve "./server"
    12 │ export * from './server';
       ╵               ~~~~~~~~~~

  > ../../node_modules/remix/browser/index.js:13:14: error: Could not resolve "./platform"
    13 │ export * from './platform';
       ╵               ~~~~~~~~~~~~


Build failed with 3 errors:
../../node_modules/remix/browser/index.js:11:14: error: Could not resolve "./client"
../../node_modules/remix/browser/index.js:12:14: error: Could not resolve "./server"
../../node_modules/remix/browser/index.js:13:14: error: Could not resolve "./platform"
💿 Built in 224ms

Without hoisting (using the hoistingLimits above), it at least spins up properly:

$ yarn workspace remix-app-template run dev
Watching Remix app in development mode...
💿 Built in 220ms
Remix App Server started at http://localhost:3000
GET / 200 - - 20.309 ms

If you switch back/forth between the hoisting rules above, you should clear out node_modules/ as some things seem to stay around with at least Yarn v3.0.1:

find . -name "node_modules" -type d -prune -exec rm -rf '{}' +

[mxro]

Thanks @rhefner - looks like it won't be too complicated to make remix work with Yarn 2/3 👍

[mihaa1]

Experiencing the same issue with yarn 3.1.1 + corepack. Just ran the tutorial:

• npx create-remix@latest
• posinstall script fails with:
  @remix-run/dev tried to access @remix-run/node, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound

[rhefner]

@mihaa1 Don't know anything about corepack, but if I'm not mistaken, that error is coming from Yarn. Have you tried updating packageExtensions in .yarnrc.yml?

For example:

packageExtensions:
  "@remix-run/dev@*":
    dependencies:
      "@remix-run/node": "*"

[silouanwright]

I can also confirm that the install does not work with Yarn 2+ Plug N Play setup. The issue is mostly around the postinstall command. Here is the first error triggered:

# This file contains the result of Yarn building a package (jokes@workspace:.)
# Script name: postinstall

@remix-run/dev tried to access @remix-run/server-runtime, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound.

Required package: @remix-run/server-runtime (via "@remix-run/server-runtime/package.json")
Required by: @remix-run/dev@npm:1.0.6 (via /Users/reydev/repos/jokes/.yarn/cache/@remix-run-dev-npm-1.0.6-73f0b55172-2fca586246.zip/node_modules/@remix-run/dev/)

Require stack:
- /Users/reydev/repos/jokes/.yarn/cache/@remix-run-dev-npm-1.0.6-73f0b55172-2fca586246.zip/node_modules/@remix-run/dev/setup.js
- /Users/reydev/repos/jokes/.yarn/cache/@remix-run-dev-npm-1.0.6-73f0b55172-2fca586246.zip/node_modules/@remix-run/dev/cli/commands.js
- /Users/reydev/repos/jokes/.yarn/cache/@remix-run-dev-npm-1.0.6-73f0b55172-2fca586246.zip/node_modules/@remix-run/dev/cli.js

When I then installed @remix-run/server-runtime, this is the error I received:

# This file contains the result of Yarn building a package (jokes@workspace:.)
# Script name: postinstall

EROFS: read-only filesystem, open '/node_modules/remix/package.json'

• The escape hatch of using nodeLinker technically works BUT I would advocate that this isn't really adding support for Yarn 2+, as this disables Plug N Play, which by default is enabled on new projects bootstrapped for Yarn 2+ (yarn init -2) and is forcing a pretty big architectural decision on a team for their entire project, that actually has some concrete benefits: https://yarnpkg.com/features/pnp. I would prefer just saying "we don't support Yarn 2+ than going the route that has us recommending us opt out of its biggest feature.

[cmd-johnson]

I dug a bit deeper into what it would take to support yarn PnP, and these are my findings so far:

TLDR: you're better off sticking to the node-modules linker for now.

• remix performs some dark magic with the remix package in the postinstall hook that will never be compatible with yarn PnP, and frankly is an anti-pattern (don't overwrite files in packages, please). I ended up throwing out remix as a dependency and now import everything directly from the @remix-run/... packages as required
• there's a few dependency/peer dependency declarations missing in the remix packages. which is an easy fix.
• after fixing the dependency declarations, you'll run into a bug in the current version of yarn, which appears to be fixed in the current canary build of yarn (yarn-3.2.0-rc.8, install it using yarn set version canary). (ref. [Bug?]: PnP fails to resolve dynamic require from CJS yarnpkg/berry#3687)
• now we run into an issue with esbuild: to make React available in all source files without having to explicitly import it, remix uses the inject option to add the contents of a react.ts file to every compiled source file. While it is possible to read files inside the zipped pnp dependencies wihle running in node, esbuild tries to read those files from an executable written in go, which fails. I added some code to compiler.ts in @remix-run/dev to copy the react.ts file contents to a temporary build folder and include that file instead. Now the build goes through.
• however, we're still not quite there yet. There's a server listening on 8002 now, but it only returns 426 Upgrade Required without any indications of what protocol it expects instead of HTTP/1.1. Looks like it doesn't start up properly, because the "Remix App Server started at ..." never gets logged to the console.

For reference, this is my current .yarnrc.yml, and these are the changes I made to the compiler.ts file to get esbuild to work

That's where I'm at now, I'll continue digging once I find some more time.

[cmd-johnson]

I made some more progress!
The build now runs fine with PnP and the dev server starts up correctly as well.
To test my changes:

Clone and build my changed version of remix:

git clone https://github.com/cmd-johnson/remix/tree/feature/yarn-pnp
cd remix
yarn install
yarn build

In the project you want to use yarn PnP in:

• switch to the current canary build of yarn: yarn set version canary. Yarn < 3.2.0 has a bug preventing remix to work with PnP.
• remove the remix dependency from your package.json, it just doesn't work with PnP right now (I'll see what I can do about that next)
• while you're at it, remove remix setup node from your postinstall script. It will fail when using PnP
• add a resolutions entry to your package.json:

    "resolutions": {
      "@remix-run/dev": "file:///path/to/the/cloned/remix/project/build/node_modules/@remix-run/dev",
      "@remix-run/express": "file:///path/to/the/cloned/remix/project/build/node_modules/@remix-run/express",
      "@remix-run/node": "file:///path/to/the/cloned/remix/project/build/node_modules/@remix-run/node",
      "@remix-run/react": "file:///path/to/the/cloned/remix/project/build/node_modules/@remix-run/react",
      "@remix-run/serve": "file:///path/to/the/cloned/remix/project/build/node_modules/@remix-run/serve"
    },

  This makes sure you're using the patched @remix-run/* versions instead of the original ones throughout your whole project
• run yarn add @yarnpkg/esbuild-plugin-pnp. This is an optional peer dependency I've added to @remix-run/dev that you'll need when using PnP for the esbuild build to go through
• run yarn add express. @remix-run/serve depends on it as a peer dependency. Depending on your setup, you might be able to get away with adding express as a peer dependency instead
• the last thing to change is fixing all imports from the remix package to imports from @remix-run/* packages. E.g.:
  • RemixBrowser → @remix-run/react/browser
  • EntryContext, MetaFunction → @remix-run/server-runtime (you might have to add a dependency to @remix-run/server-runtime to your project as well)
  • Most other dependencies appear to be exported by @remix-run/react

After that, you should be able to start the dev server using yarn dev as usual.

I hope I didn't forget anything. I'll draft a PR once I get around to get the remix package to work with PnP.