Node required to be installed outside of Bazel

[promiseofcake]

It seems like node is externally required (installed outside of Bazel) to process '@yarn_modules//' targets. Is there a way we can use a fetched node for this similarly to as we do for go-related builds in bazelbuild/rules_go?

[pcj]

Can you elaborate? Node should be installed automatically. I'm not aware of how you're using @yarn_modules// that would require a node install outside of bazel.

[promiseofcake]

Sure, so I have node installed via nvm on my machine (for development outside of Bazel), and when I removed it I started having issues with my builds. Here is how we are using this repo now:

# WORKSPACE
git_repository(
    name = "org_pubref_rules_node",
    commit = "2b4b38dcb76eba98419c932c98bd22fe6e322c9f",
    remote = "https://github.com/pubref/rules_node.git",
)

git_repository(
    name = "org_pubref_rules_protobuf",
    remote = "https://github.com/pubref/rules_protobuf",
    commit = "563b674a2ce6650d459732932ea2bc98c9c9a9bf", # Nov 28, 2017
)

load("@org_pubref_rules_node//node:rules.bzl", "node_repositories", "yarn_modules")
node_repositories()

# required to initialize dependencies previously out of our control via @org_pubref_rules_node
yarn_modules(
    name = "yarn_modules",
    deps = {
        "google-protobuf": "3.4.0",
        "grpc": "1.6.0"
    },
)
node_proto_repositories()

# BUILD
node_proto_library(
    name = "foojs_grpc",
    imports = ["protobufs/imports"],
    protos = ["protos/foo/foo.proto"],
    verbose = 1,
    with_grpc = True,
)

node_binary(
    name = "foojs",
    main = "index.js",
    deps = [
        ":foojs_grpc",
        "@yarn_modules//:_all_",
    ],
)

Output

bazel build //foo/js:foojs
...........
DEBUG: /private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/bazel_tools/tools/build_defs/pkg/pkg.bzl:197:9: @//foo/js:foojs_bundle: you provided a non dictionary to the pkg_tar `files` attribute. This attribute was renamed to `srcs`. Consider renaming it in your BUILD file.
ERROR: /path/to/foo/js/BUILD:3:1: no such package '@yarn_modules//': Traceback (most recent call last):
    File "/private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 59
        execute(ctx, [node, yarn_js, "install"], qui...)
    File "/private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 4, in execute
        fail((" ".join(cmds) + ("failed: \nST...))))
/private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/node/bin/node /private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/yarn/bin/yarn.js installfailed:
STDOUT:
yarn install v1.0.1
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

STDERR:
warning package.json: No license field
warning yarn_modules@1.0.0: No license field
error /private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/yarn_modules/node_modules/grpc: Command failed.
Exit code: 127
Command: ./node_modules/.bin/node-pre-gyp install --fallback-to-build --library=static_library
Arguments:
Directory: /private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/yarn_modules/node_modules/grpc
Output:
env: node: No such file or directory
 and referenced by '//foo/js:foojs'
ERROR: Analysis of target '//foo/js:foojs' failed; build aborted: no such package '@yarn_modules//': Traceback (most recent call last):
    File "/private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 59
        execute(ctx, [node, yarn_js, "install"], qui...)
    File "/private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 4, in execute
        fail((" ".join(cmds) + ("failed: \nST...))))
/private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/node/bin/node /private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/yarn/bin/yarn.js installfailed:
STDOUT:
yarn install v1.0.1
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

STDERR:
warning package.json: No license field
warning yarn_modules@1.0.0: No license field
error /private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/yarn_modules/node_modules/grpc: Command failed.
Exit code: 127
Command: ./node_modules/.bin/node-pre-gyp install --fallback-to-build --library=static_library
Arguments:
Directory: /private/var/tmp/_bazel_lucas/4e96d1a6d41518319f74f637163517c2/external/yarn_modules/node_modules/grpc
Output:
env: node: No such file or directory
INFO: Elapsed time: 21.171s
FAILED: Build did NOT complete successfully (7 packages loaded)

Relink Node via NVM

export NVM_DIR="/Users/lucas/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"

Then it succeeds.

[promiseofcake]

Also attempting to just build the yarn_module targets I get the same error:

bazel build @yarn_modules//:_all_
...........
ERROR: Skipping '@yarn_modules//:_all_': no such package '@yarn_modules//': Traceback (most recent call last):
	File "/private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 59
		execute(ctx, [node, yarn_js, "install"], qui...)
	File "/private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 4, in execute
		fail((" ".join(cmds) + ("failed: \nST...))))
/private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/node/bin/node /private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/yarn/bin/yarn.js installfailed:
STDOUT:
yarn install v1.0.1
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

STDERR:
warning package.json: No license field
warning yarn_modules@1.0.0: No license field
error /private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/yarn_modules/node_modules/grpc: Command failed.
Exit code: 127
Command: ./node_modules/.bin/node-pre-gyp install --fallback-to-build --library=static_library
Arguments:
Directory: /private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/yarn_modules/node_modules/grpc
Output:
env: node: No such file or directory
WARNING: Target pattern parsing failed.
ERROR: no such package '@yarn_modules//': Traceback (most recent call last):
	File "/private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 59
		execute(ctx, [node, yarn_js, "install"], qui...)
	File "/private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/org_pubref_rules_node/node/internal/yarn_modules.bzl", line 4, in execute
		fail((" ".join(cmds) + ("failed: \nST...))))
/private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/node/bin/node /private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/yarn/bin/yarn.js installfailed:
STDOUT:
yarn install v1.0.1
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

STDERR:
warning package.json: No license field
warning yarn_modules@1.0.0: No license field
error /private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/yarn_modules/node_modules/grpc: Command failed.
Exit code: 127
Command: ./node_modules/.bin/node-pre-gyp install --fallback-to-build --library=static_library
Arguments:
Directory: /private/var/tmp/_bazel_lucas/4e96d1a6d41460169f74f268073930c2/external/yarn_modules/node_modules/grpc
Output:
env: node: No such file or directory
INFO: Elapsed time: 15.231s
FAILED: Build did NOT complete successfully (0 packages loaded)

[promiseofcake]

Hm maybe the issue is one of the transitive dependencies from the yarn_modules has a node command as part of their instal stanza in their package.json, related to #40?

[pcj]

OK I see yeah that looks like a bug. The node-pre-gyp or related must have a shebang script line line #!/usr/bin/env node. We need to modify the PATH to prepend the bazel-installed one prior to calling the node yarn.js part.

I'll plan to fix this later this afternoon or tonight unless you want to do it first.

[pcj]

Same symptom as #40 but in a different place.

[pcj]

Looks like you're on darwin, yes? Unable to repro on my linux laptop.

[promiseofcake]

Yup Darwin, hm, is there a place I can should look to try to add debugging to try to figure out what is going on?

[promiseofcake]

@pcj -- so I think the problem was I was using a hash that did not have #41 merged in. I was using2b4b38dcb76eba98419c932c98bd22fe6e322c9f, which is the linked hash from the HEAD of pubref/rules_protobuf. I will bump my rules_node version and it should then be solved, perhaps if nothing is blocking I will modify pubref/rules_protobuf to reference that latest hash as well.