Optional dependencies still have problem

[TrySound]

Do you want to request a feature or report a bug?

Bug

What is the current behavior?

Optional dependency is excluded but still try to build.

warning fsevents@1.0.14: The platform "win32" is incompatible with this module.
info "fsevents@1.0.14" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
[1/1] ⠁ fsevents: gyp info using node@4.5.0 | win32 | x64
[-/1] ⠁ waiting...
[-/1] ⠁ waiting...
[-/1] ⠁ waiting...
error Error running install script for optional dependency: "[...]\\node_modules\\fsevents

If the current behavior is a bug, please provide the steps to reproduce.

yarn add chokidar

What is the expected behavior?

Incompatible optional dependency should be excluded completly.

Please mention your node.js, yarn and operating system version.

node4
yarn0.16.1
windows7

[omggga]

Same problem here. Node 6.5, win 7

info "fsevents@1.0.14" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...    
[4/4] Building fresh packages...
[1/1] ? fsevents: File "%AppData%\Roaming\nvm\v6.5.0\node_modules\yarn\node_modules\node-gyp\gyp\py
[-/1] ? waiting...
[-/1] ? waiting...
[-/1] ? waiting...
error Error running install script for optional dependency: "C:\\Projects\\pcab-frontend\\node_modules\\fsevents: Command failed

[Luddinus]

Same issue here (windows 7)

[m59peacemaker]

I am ending up with a lot more MB in node_modules installing with yarn. I am certain that one big factor is fsevents when installing chokidar, same as the OP is facing. I am on ubuntu 16.04, node 6.2.2. yarn seems to install all optional dependencies, regardless of platform support.

[moersing]

Same issue on win10 and yarn version is 0.16.1, also resemble error Not compatible with your operating system or architecture: fsevents@1.0.14 on npm.

[Daniel15]

This should have been fixed by #525, but it looks like it regressed at some point 😢 . You can replicate the issue on a non-Windows environment by running yarn add @daniel15/win-test-2 (edit: sorry, I was incorrect)

[wyze]

@Daniel15, I don't think that module works as it specifies the os field so it errors on the main package (for non-Windows users), before it gets to the optionalDependencies.

But if I exclude the platform check for win-test-2 explicitly, this is the output I get:

$ yarn add @daniel15/win-test-2
info No lockfile found.
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
warning @daniel15/win-test@0.0.1: The platform "darwin" is incompatible with this module.
info "@daniel15/win-test@0.0.1" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
success Saved 2 new dependencies.
├─ @daniel15/win-test-2@0.0.1
└─ @daniel15/win-test@0.0.1
✨  Done in 1.68s.

Verified the optionalDependency module was installed too:

$ ls node_modules/@daniel15
win-test   win-test-2

It is interesting that it reports excluding it from installation, but installs it anyways.

[Daniel15]

@wyze - Oops, my mistake! Sorry for the misleading comment. I just realised my test package was configured incorrectly.

[kriscarle]

In case it helps, running yarn install --production resolved the fsevents error for me on linux using version 0.16.1. With the production flag I just get a warning that it is skipping fsevents.

[billiegoose]

+1

> yarn --version
0.16.1

Expected behavior:

> yarn add chokidar
yarn add v0.16.1
[1/4] Resolving packages...
[2/4] Fetching packages...
warning fsevents@1.0.15: The platform "win32" is incompatible with this module.
info "fsevents@1.0.15" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
node-pre-success Saved lockfile.
success Saved XX new dependencies
....
Done in 13.22s.

Actual behavior:

> yarn add chokidar
yarn add v0.16.1
[1/4] Resolving packages...
[2/4] Fetching packages...
warning fsevents@1.0.15: The platform "win32" is incompatible with this module.
info "fsevents@1.0.15" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
[1/1] ⢀ fsevents: node -v v5.3.0
[-/1] ⢀ waiting...
[-/1] ⢀ waiting...
[-/1] ⢀ waiting...
error Error running install script for optional dependency: "E:\\tmp\\node_modules\\fsevents: Command failed.\nExit code: 1\nCommand: C:\\WINDOWS\\system32\\cmd.exe\nArguments: /d /s /c node-pre-gyp install --fallback-to-build\nDirectory: E:\\tmp\\node_modules\\fsevents\nOutput:\nnode-pre-gyp info it worked if it ends with ok\nnode-pre-gyp info using node-pre-gyp@0.6.29\nnode-pre-gyp info using node@5.3.0 | win32 | x64\nnode-pre-gyp info check checked for \"E:\\tmp\\node_modules\\fsevents\\lib\\binding\\Release\\node-v47-win32-x64\\fse.node\" (not found)\nnode-pre-gyp http GET https://fsevents-binaries.s3-us-west-2.amazonaws.com/v1.0.15/fse-v1.0.15-node-v47-win32-x64.tar.gz\nnode-pre-gyp http 404 https://fsevents-binaries.s3-us-west-2.amazonaws.com/v1.0.15/fse-v1.0.15-node-v47-win32-x64.tar.gz\nnode-pre-gyp ERR! Tried to download: https://fsevents-binaries.s3-us-west-2.amazonaws.com/v1.0.15/fse-v1.0.15-node-v47-win32-x64.tar.gz \nnode-pre-gyp ERR! Pre-built binaries not found for fsevents@1.0.15 and node@5.3.0 (node-v47 ABI) (falling back to source compile with node-gyp) \nnode-pre-gyp http Pre-built binary not available for your system, looked for https://fsevents-binaries.s3-us-west-2.amazonaws.com/v1.0.15/fse-v1.0.15-node-v47-win32-x64.tar.gz \n\r\nE:\\tmp\\node_modules\\fsevents>if not defined npm_config_node_gyp (node \"E:\\global_modules\\node_modules\\yarn\\bin\\node-gyp-bin\\\\..\\..\\node_modules\\node-gyp\\bin\\node-gyp.js\" clean )  else (node  clean ) \r\ngyp info it worked if it ends with ok\ngyp info using node-gyp@3.4.0\ngyp info using node@5.3.0 | win32 | x64\ngyp info ok \n\r\nE:\\tmp\\node_modules\\fsevents>if not defined npm_config_node_gyp (node \"E:\\global_modules\\node_modules\\yarn\\bin\\node-gyp-bin\\\\..\\..\\node_modules\\node-gyp\\bin\\node-gyp.js\" configure --fallback-to-build --module=E:\\tmp\\node_modules\\fsevents\\lib\\binding\\Release\\node-v47-win32-x64\\fse.node --module_name=fse --module_path=E:\\tmp\\node_modules\\fsevents\\lib\\binding\\Release\\node-v47-win32-x64 )  else (node  configure --fallback-to-build --module=E:\\tmp\\node_modules\\fsevents\\lib\\binding\\Release\\node-v47-win32-x64\\fse.node --module_name=fse --module_path=E:\\tmp\\node_modules\\fsevents\\lib\\binding\\Release\\node-v47-win32-x64 ) \r\ngyp info it worked if it ends with ok\ngyp info using node-gyp@3.4.0\ngyp info using node@5.3.0 | win32 | x64\ngyp info spawn C:\\Python27\\python.EXE\ngyp info spawn args [ 'E:\\\\global_modules\\\\node_modules\\\\yarn\\\\node_modules\\\\node-gyp\\\\gyp\\\\gyp_main.py',\ngyp info spawn args   'binding.gyp',\ngyp info spawn args   '-f',\ngyp info spawn args   'msvs',\ngyp info spawn args   '-G',\ngyp info spawn args   'msvs_version=auto',\ngyp info spawn args   '-I',\ngyp info spawn args   'E:\\\\tmp\\\\node_modules\\\\fsevents\\\\build\\\\config.gypi',\ngyp info spawn args   '-I',\ngyp info spawn args   'E:\\\\global_modules\\\\node_modules\\\\yarn\\\\node_modules\\\\node-gyp\\\\addon.gypi',\ngyp info spawn args   '-I',\ngyp info spawn args   'C:\\\\Users\\\\Will\\\\.node-gyp\\\\5.3.0\\\\include\\\\node\\\\common.gypi',\ngyp info spawn args   '-Dlibrary=shared_library',\ngyp info spawn args   '-Dvisibility=default',\ngyp info spawn args   '-Dnode_root_dir=C:\\\\Users\\\\Will\\\\.node-gyp\\\\5.3.0',\ngyp info spawn args   '-Dnode_gyp_dir=E:\\\\global_modules\\\\node_modules\\\\yarn\\\\node_modules\\\\node-gyp',\ngyp info spawn args   '-Dnode_lib_file=node.lib',\ngyp info spawn args   '-Dmodule_root_dir=E:\\\\tmp\\\\node_modules\\\\fsevents',\ngyp info spawn args   '--depth=.',\ngyp info spawn args   '--no-parallel',\ngyp info spawn args   '--generator-output',\ngyp info spawn args   'E:\\\\tmp\\\\node_modules\\\\fsevents\\\\build',\ngyp info spawn args   '-Goutput_dir=.' ]\nTraceback (most recent call last):\r\n  File \"E:\\global_modules\\node_modules\\yarn\\node_modules\\node-gyp\\gyp\\gyp_main.py\", line 16, in <module>\r\n    sys.exit(gyp.script_main())\r\n  File \"E:\\global_modules\\node_modules\\yarn\\node_modules\\node-gyp\\gyp\\pylib\\gyp\\__init__.py\", line 545, in script_main\r\n    return main(sys.argv[1:])\r\n  File \"E:\\global_modules\\node_modules\\yarn\\node_modules\\node-gyp\\gyp\\pylib\\gyp\\__init__.py\", line 538, in main\r\n    return gyp_main(args)\r\n  File \"E:\\global_modules\\node_modules\\yarn\\node_modules\\node-gyp\\gyp\\pylib\\gyp\\__init__.py\", line 523, in gyp_main\r\n    generator.GenerateOutput(flat_list, targets, data, params)\r\n  File \"E:\\global_modules\\node_modules\\yarn\\node_modules\\node-gyp\\gyp\\pylib\\gyp\\generator\\msvs.py\", line 2003, in GenerateOutput\r\n    sln_projects, project_objects, flat=msvs_version.FlatSolution())\r\n  File \"E:\\global_modules\\node_modules\\yarn\\node_modules\\node-gyp\\gyp\\pylib\\gyp\\generator\\msvs.py\", line 1762, in _GatherSolutionFolders\r\n    return _DictsToFolders('', root, flat)\r\n  File \"E:\\global_modules\\node_modules\\yarn\\node_modules\\node-gyp\\gyp\\pylib\\gyp\\generator\\msvs.py\", line 1715, in _DictsToFolders\r\n    for folder, contents in bucket.iteritems():\r\nAttributeError: 'MSVSProject' object has no attribute 'iteritems'\r\ngyp ERR! configure error \ngyp ERR! stack Error: `gyp` failed with exit code: 1\ngyp ERR! stack     at ChildProcess.onCpExit (E:\\global_modules\\node_modules\\yarn\\node_modules\\node-gyp\\lib\\configure.js:305:16)\ngyp ERR! stack     at emitTwo (events.js:87:13)\ngyp ERR! stack     at ChildProcess.emit (events.js:172:7)\ngyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)\ngyp ERR! System Windows_NT 10.0.14393\ngyp ERR! command \"C:\\\\PROGRAM FILES\\\\NODEJS\\\\node.exe\" \"E:\\\\global_modules\\\\node_modules\\\\yarn\\\\node_modules\\\\node-gyp\\\\bin\\\\node-gyp.js\" \"configure\" \"--fallback-to-build\" \"--module=E:\\\\tmp\\\\node_modules\\\\fsevents\\\\lib\\\\binding\\\\Release\\\\node-v47-win32-x64\\\\fse.node\" \"--module_name=fse\" \"--module_path=E:\\\\tmp\\\\node_modules\\\\fsevents\\\\lib\\\\binding\\\\Release\\\\node-v47-win32-x64\"\ngyp ERR! cwd E:\\tmp\\node_modules\\fsevents\ngyp ERR! node -v v5.3.0\ngyp ERR! node-gyp -v v3.4.0\ngyp ERR! not ok \nnode-pre-gyp ERR! build error \nnode-pre-gyp ERR! stack Error: Failed to execute 'node-gyp.cmd configure --fallback-to-build --module=E:\\tmp\\node_modules\\fsevents\\lib\\binding\\Release\\node-v47-win32-x64\\fse.node --module_name=fse --module_path=E:\\tmp\\node_modules\\fsevents\\lib\\binding\\Release\\node-v47-win32-x64' (1)\nnode-pre-gyp ERR! stack     at ChildProcess.<anonymous> (E:\\tmp\\node_modules\\fsevents\\node_modules\\node-pre-gyp\\lib\\util\\compile.js:83:29)\nnode-pre-gyp ERR! stack     at emitTwo (events.js:87:13)\nnode-pre-gyp ERR! stack     at ChildProcess.emit (events.js:172:7)\nnode-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:818:16)\nnode-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)\nnode-pre-gyp ERR! System Windows_NT 10.0.14393\nnode-pre-gyp ERR! command \"C:\\\\PROGRAM FILES\\\\NODEJS\\\\node.exe\" \"E:\\\\tmp\\\\node_modules\\\\fsevents\\\\node_modules\\\\node-pre-gyp\\\\bin\\\\node-pre-gyp\" \"install\" \"--fallback-to-build\"\nnode-pre-success Saved lockfile.
success Saved 0 new dependencies.
Done in 12.34s.

Repeated installation attempts eventually makes it happy, but clearing out node_modules and package.json and doing a fresh install of chokidar reliably fails.

Update:
It looks to me like what is happening is that PackageCompatibility.check is merely printing a warning for optionalDependencies that fail the validation check. It is not actively removing them from the queue so PackageInstallScripts.init tries to run the install script for fsevents anyway.

@kriscarle I think that might be a fluke. I tried installing chokidar with the production flag and it still failed.