Yarn 'PnP' functionality does not work on windows: You cannot require a package that is not declared in your dependencies

[nicewaytodoit]

Currently installed:
node version: v10.15.0
yarn version: 1.12.3
react-scripts: 2.1.2
Windows 10 x64

I have tried few things to enable PnP but neither works. First I've tried enabling it on existing project and it did not work. Then installing it as new react app with:

yarn create react-app hello-yarn --use-pnp

But it still created node_module files.
After I deleted node_module files, and tried running yarn --enable-pnp cash was created but at the same time there was V3 and V4 folder.
Still no luck. Then deleted Cache and configured yarn config set cache-folder /c/proj/YarnCache
So config file looks like this:

info yarn config
{ 'version-tag-prefix':
   'v',
  'version-git-tag':
   true,
  'version-commit-hooks':
   true,
  'version-git-sign':
   false,
  'version-git-message':
   'v%s',
  'init-version':
   '1.0.0',
  'init-license':
   'MIT',
  'save-prefix':
   '^',
  'bin-links':
   true,
  'ignore-scripts':
   false,
  'ignore-optional':
   false,
  registry:
   'https://registry.yarnpkg.com',
  'strict-ssl':
   true,
  'user-agent':
   'yarn/1.12.3 npm/? node/v10.15.0 win32 x64',
  'cache-folder':
   'C:\\proj\\YarnCache',
  lastUpdateCheck:
   1546153614636 }

(by the way I do not have a clue why is JSON config looks all unaligned and borken)

Went again into the project and again run yarn --enable-pnp
Checked the cache, it is created successfully with all hash file keys within folder 'C:\proj\YarnCache',

When I run project yarn start I get following error

C:\proj\Test\hello-yarn-pnp\.pnp.js:15511
    throw firstError;
    ^
**Error: You cannot require a package ("C:\proj\YarnCache\v4\npm-react-scripts-2.1.2-cf1686cd9ca3c6148e1428e97
aac12995f01b554\node_modules\react-scripts\bin\react-scripts.js") that is not declared in your dependencies**
(via "C:\proj\Test\hello-yarn-pnp/")
    at makeError (C:\proj\Test\hello-yarn-pnp\.pnp.js:55:17)
    at Object.resolveToUnqualified (C:\proj\Test\hello-yarn-pnp\.pnp.js:15244:17)
    at Object.resolveRequest (C:\proj\Test\hello-yarn-pnp\.pnp.js:15322:31)
    at Function.Module._resolveFilename (C:\proj\Test\hello-yarn-pnp\.pnp.js:15502:30)
    at Function.Module._load (C:\proj\Test\hello-yarn-pnp\.pnp.js:15420:31)
    at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
    at startup (internal/bootstrap/node.js:283:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:743:3)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Additionally I have tried deleting yarn.lock, .pnp file and folder and running again yarn -pnp everything went smoothly but again without luck, I am getting the same dependency error.

To emphasize: Project has been created with yarn crate react-app --use-pnp
If I remove pnp option and install packages in standard way project will run perfectly fine.

What is the expected behavior?

I expect project to run without errors :)

Does anyone has a clue what is going on and is it yarn issue or my system configuration issue?

[Daniel15]

@arcanis Do you know if there's known issues with PnP on Windows?

[arcanis]

No known issue, our testsuite runs on Windows as well 🤔

The error in this issue seems to point towards a regexp detecting C:\proj\YarnCache\v4\npm-react-scripts-2.1.2-cf1686cd9ca3c6148e1428e97 aac12995f01b554\node_modules\react-scripts\bin\react-scripts.js as a package name rather than an absolute path, which is something that's been solved a few releases ago

[nicewaytodoit]

Is it possible that there is some kind of clash between yarn installed trough NPM and yarn installed with windows executable?
Can I help you by giving more logs or do you want me to run something on my machine?

[arcanis]

@nicewaytodoit can you share your .pnp.js file? That could help figuring out the problem

[nicewaytodoit]

Yes, you have git repository with entire project:
Clone the following repo:
Yarn-pnp-error repo

https://github.com/nicewaytodoit/yarn-pnp-error
I have put project on C drive :
c:\proj\Test\yarn-pnp-error
And Yarn cashe is located:
C:\proj\YarnCache
(Just so you can replicate exactly what I have done)
react-scripts has changed version to : 2.1.3

During installation with --use-pnp this time I got error:

So this:
yarn create react-app hello-yarn --use-pnp
does not work anymore error is:

Aborting installation.
node --require C:\proj\Test\yarn-pnp-error.pnp.js has failed.

C:\proj\Test\yarn-pnp-error\.pnp.js:15590
    throw firstError;
    ^
Error: Package "react-scripts@2.1.3" (via "C:\proj\YarnCache\v4\npm-react-scripts-2.1.3-6e49be279f4039fb9f330d2b3529b933b8e90945\node_modul
es\react-scripts\config\paths.js") is trying to require the package "C:\proj\Test\yarn-pnp-error\package.json" (via "C:\proj\Test\yarn-pnp-
error\package.json") without it being listed in its dependencies (@babel/core, @svgr/webpack, babel-core, babel-eslint, babel-jest, babel-l
oader, babel-plugin-named-asset-import, babel-preset-react-app, bfj, case-sensitive-paths-webpack-plugin, chalk, css-loader, dotenv, dotenv
-expand, eslint, eslint-config-react-app, eslint-loader, eslint-plugin-flowtype, eslint-plugin-import, eslint-plugin-jsx-a11y, eslint-plugi
n-react, file-loader, fork-ts-checker-webpack-plugin-alt, fs-extra, html-webpack-plugin, identity-obj-proxy, jest, jest-pnp-resolver, jest-
resolve, mini-css-extract-plugin, optimize-css-assets-webpack-plugin, pnp-webpack-plugin, postcss-flexbugs-fixes, postcss-loader, postcss-p
reset-env, postcss-safe-parser, react-app-polyfill, react-dev-utils, resolve, sass-loader, style-loader, terser-webpack-plugin, url-loader,
 webpack, webpack-dev-server, webpack-manifest-plugin, workbox-webpack-plugin, react-scripts)

What succeeded is following:
yarn create react-app yarn-pnp-error

and then
yarn --enable-pnp

You have .pnp folder .pnp.js file.

And bug is there still present.

Now I have noticed that:

c:\proj\Test\yarn-pnp-error\.pnp\externals\pnp-3ad4f0ca2cef43bdcc7717813f8802e19cbf0bdc\node_modules\@babel\plugin-transform-async-to-generator\.yarn-metadata.json

_loc property is point to wrong cache ?

C:\\Users\\mind\\Desktop\\Job\\React\\YarnCache\\v4\\npm-@babel-plugin-transform-async-to-generator-7.2.0-68b8a438663e88519e65b776f8938f3445b1a2ff\\node_modules\\@babel\\plugin-transform-async-to-generator\\package.json

If yarn config list is giving 'cache-folder': 'C:\\proj\\YarnCache', why is that .yarn-metadata.json pointing to some other folder and where is that pointer stored ?

[nicewaytodoit]

Just additional info, just now I tried couple things and behavior is really strange:
In the same project I have tried yarn cash clear then manually deleted other cash folder _loc property is pointing.
Then I tried running yarn --enable-pnp but .pnp folder/file are not recreated.
Then I deleted yarn.lock node_module folder and when i run yarn --enable-pnp
Instead of creating .pnp and .pnp.js yarn recreates node_modules and yarn.lock file with all packages like normal yarn command would do.

Config file is still pointing to 'C:\proj\YarnCache' and packages there are intact.

tried again :

$ yarn cache clean
yarn cache v1.12.3
warning Plug'n'Play on Windows doesn't support the cache and project to be kept on separate drives

and folder 'C:\proj\YarnCache' is empty ...
then again yarn --enable-pnp
2 step Fetching packages recreates Cache ...
3 step Linking Dependencies node_module appears but full size (160MB), but there is no .pnp.js or .pnp folder.

Either I am missing something or this now does not work at all ?!

[nicewaytodoit]

Now I have tried following:

• npm uninstall -g yarn
• Windows uninstall yarn
• deleted c:/proj/YarnCache
• deleted .yarnrc
• deleted /AppData/Local/Yarn
• Installed only msi package yarn installation 1.12.3
• went to folder with project rm -rf node_modules and rm yarn.lock
• yarn config set cache-folder /c/proj/YarnCache/
• yarn config list
• cache-folder set to 'C:\proj\YarnCache'
• in project folder turn yarn --enable-pnp
• Cash recreated but there is no .pnp.js or .pnp/ folder, node_modules restored to full size (~120Mb)

PNP is not working ...

[arcanis]

tl;dr Please upgrade to Yarn 1.13

Interesting - your .pnp.js file has been generated with the wrong Yarn version (I really should put a --version on this file or something). Take a look at the correct regex in our source code:

https://github.com/yarnpkg/yarn/blob/master/src/util/generate-pnp-map-api.tpl.js#L37

By comparison, here's the one found in your .pnp.js:

https://github.com/nicewaytodoit/yarn-pnp-error/blob/master/.pnp.js#L37

As you can see, the regex in your file doesn't support Windows paths. This has been fixed in 7f41910, which got released with the 1.13 (my bad, I forgot to include it in the changelog, hence why I didn't suggest you to upgrade before).

[nicewaytodoit]

I have installed version yarn 1.13 and at first attempt it is not working.
Again yarn --enable-pnp will install node_modules full size not recreating .pnp.js and .pnp folders.

Before I give up the last thing I am trying is to remove:

 "installConfig": {
    "pnp": true
  }

from package.json.

And.... no luck!
Basically, in existing project yarn will never recreate .pnp.js file and .pnp folder.
In most of cases people will not want to create application from scratch, they will want to enable pnp in existing project. Currently that does not work in windows.
Can you tell me what do I need to do in order to recreate .pnp.js and pnp folder ?

What about installing from scratch? Tried that as well yarn create react-app hello-yarn --use-pnp
but at a hint of server running and raising my smile i got this:

C:\proj\YarnCache\v4\npm-pnp-webpack-plugin-1.1.0-947a96d1db94bb5a1fc014d83b581e
428699ac8c\node_modules\pnp-webpack-plugin\index.js:69
        throw new Error(`Cannot successfully resolve this dependency - issuer no
t supported (${issuer})`);
        ^

Error: Cannot successfully resolve this dependency - issuer not supported (C:\pr
oj\Test\yarn-test-pnp-2\src\index.js)
    at resolver.getHook.tapAsync (C:\proj\YarnCache\v4\npm-pnp-webpack-plugin-1.
1.0-947a96d1db94bb5a1fc014d83b581e428699ac8c\node_modules\pnp-webpack-plugin\ind
ex.js:69:15)
    at AsyncSeriesBailHook.eval [as callAsync] (eval at create (C:\proj\YarnCach
e\v4\npm-tapable-1.1.1-4d297923c5a72a42360de2ab52dadfaaec00018e\node_modules\tap
able\lib\HookCodeFactory.js:32:10), <anonymous>:7:1)
    at AsyncSeriesBailHook.lazyCompileHook (C:\proj\YarnCache\v4\npm-tapable-1.1
.1-4d297923c5a72a42360de2ab52dadfaaec00018e\node_modules\tapable\lib\Hook.js:154
:20)
    at Resolver.doResolve (C:\proj\YarnCache\v4\npm-enhanced-resolve-4.1.0-41c7e
0bfdfe74ac1ffe1e57ad6a5c6c9f3742a7f\node_modules\enhanced-resolve\lib\Resolver.j
s:235:16)
    at resolver.getHook.tapAsync (C:\proj\YarnCache\v4\npm-enhanced-resolve-4.1.
0-41c7e0bfdfe74ac1ffe1e57ad6a5c6c9f3742a7f\node_modules\enhanced-resolve\lib\Try
NextPlugin.js:17:13)
    at AsyncSeriesBailHook.eval [as callAsync] (eval at create (C:\proj\YarnCach
e\v4\npm-tapable-1.1.1-4d297923c5a72a42360de2ab52dadfaaec00018e\node_modules\tap
able\lib\HookCodeFactory.js:32:10), <anonymous>:7:1)
    at AsyncSeriesBailHook.lazyCompileHook (C:\proj\YarnCache\v4\npm-tapable-1.1
.1-4d297923c5a72a42360de2ab52dadfaaec00018e\node_modules\tapable\lib\Hook.js:154
:20)
    at Resolver.doResolve (C:\proj\YarnCache\v4\npm-enhanced-resolve-4.1.0-41c7e
0bfdfe74ac1ffe1e57ad6a5c6c9f3742a7f\node_modules\enhanced-resolve\lib\Resolver.j
s:235:16)
    at resolver.getHook.tapAsync (C:\proj\YarnCache\v4\npm-enhanced-resolve-4.1.
0-41c7e0bfdfe74ac1ffe1e57ad6a5c6c9f3742a7f\node_modules\enhanced-resolve\lib\Mod
uleKindPlugin.js:19:13)
    at _fn41 (eval at create (C:\proj\YarnCache\v4\npm-tapable-1.1.1-4d297923c5a
72a42360de2ab52dadfaaec00018e\node_modules\tapable\lib\HookCodeFactory.js:32:10)
, <anonymous>:385:1)
    at resolver.getHook.tapAsync (C:\proj\YarnCache\v4\npm-enhanced-resolve-4.1.
0-41c7e0bfdfe74ac1ffe1e57ad6a5c6c9f3742a7f\node_modules\enhanced-resolve\lib\Ali
asPlugin.js:53:11)
    at _fn40 (eval at create (C:\proj\YarnCache\v4\npm-tapable-1.1.1-4d297923c5a
72a42360de2ab52dadfaaec00018e\node_modules\tapable\lib\HookCodeFactory.js:32:10)
, <anonymous>:376:1)
    at resolver.getHook.tapAsync (C:\proj\YarnCache\v4\npm-enhanced-resolve-4.1.
0-41c7e0bfdfe74ac1ffe1e57ad6a5c6c9f3742a7f\node_modules\enhanced-resolve\lib\Ali
asPlugin.js:53:11)
    at _fn39 (eval at create (C:\proj\YarnCache\v4\npm-tapable-1.1.1-4d297923c5a
72a42360de2ab52dadfaaec00018e\node_modules\tapable\lib\HookCodeFactory.js:32:10)
, <anonymous>:367:1)
    at resolver.getHook.tapAsync (C:\proj\YarnCache\v4\npm-enhanced-resolve-4.1.
0-41c7e0bfdfe74ac1ffe1e57ad6a5c6c9f3742a7f\node_modules\enhanced-resolve\lib\Ali
asPlugin.js:53:11)
    at _fn38 (eval at create (C:\proj\YarnCache\v4\npm-tapable-1.1.1-4d297923c5a
72a42360de2ab52dadfaaec00018e\node_modules\tapable\lib\HookCodeFactory.js:32:10)
, <anonymous>:358:1)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this comm
and.

So, no PNP does not work.

[arcanis]

This installConfig part is what enable PnP or not. When running --enable-pnp, it simply adds this field then runs the regular install. As long as this field is there, the PnP file will (should?) be regenerated. Conversely, if this field isn't there, then running yarn install won't generate the PnP file (and that's expected).

Anyway, what I'm not sure is why the --enable-pnp flag doesn't work (since if I understand correctly, the .pnp.js isn't generated, correct?). It's the only report I got of this so I think there's a parameter we're missing, but I can't see which one 😔

[nicewaytodoit]

Sorry just updated my previous comment please could you check it out.
Before running --enable-pnp I have stripped project from node_moduls, yarn.lock, .pnp.js, .pnp folder and removed installConfig from package json.
Completely "naked" project. And yarn --enable-pnp will just behave like running yarn install with exception that it will populate V4 cache files first and then do the copy to node_modules. It will not recreate .pnp.js or .pnp folder.

[arcanis]

Ok so this issuer not supported error comes from pnp-webpack-plugin. In particular, it is emitted when path.isAbsolute("C:\\proj\\Test\\yarn-test-pnp-2\\src\\index.js") returns false. Since you're obviously on a Windows system this shouldn't happen!

By any chance, are you using a system like cygwin? If so, can you try running the following command and report what it says?

$> node
> process.platform
> require('path') === require('path').posix
> require('path') === require('path').win32

[nicewaytodoit]

I am trying all this with git-bash (linux shell) so may be there is issue with winpty.
Outputs are:

$ node
> process.platform
'win32'
> require('path') === require('path').posix
false
> require('path') === require('path').win32
true

[nicewaytodoit]

Quick update VS Code bash failed same way CMD shell as well.
Same ouput for requested code.

You, know what? Let me debug this forking thing, so I can see what is going on and I will update you.