$(location) should use runtime path when used in jvm_flags attribute

[shahms]

We have a binary which requires the javac.jar be present in its bootclasspath. Prior to bazel 0.4.4, the following invocation worked (but was obviously less than ideal):

jvm_flags = [
    "-Xbootclasspath/p:$$JAVA_RUNFILES/bazel_tools/third_party/java/jdk/langtools/javac.jar",
],

With bazel 0.4.4 the file name specific path has broken, as the actual file name of the target is now javac-9-dev-r3297-1.jar.

Ideally, we'd be able to use $(location @bazel_tools//third_party/java/jdk/langtools:javac_jar), however that expands to external/bazel_tools/third_party/java/jdk/langtools/javac-9-dev-r3297-1.jar which is the build path and not the runtime path for that file. Needless to say, I'd like to avoid hard-coding another path which will break or fragile inline path-munging hacks. It seems like the correct answer is for $(location) to use the runtime path when specified in jvm_flags.

[meteorcloudy]

@damienmg Do you think it's reasonable to support this feature? Or is there a better way to find the runtime path of javac.jar?

[meteorcloudy]

//cc @kchodorow

[damienmg]

FWIW we cannot make $(location ) returns an absolute path but it should be able to return a path relative to the runfile tree. @kchodorow should definitely knows how / when it will be possible to do so.

[kchodorow]

It's an awfully big hammer, but Skylark's short_path gives you the runtime path. We're working on making the runtime path match the build path (for sources, at least), but it's taking a lot longer than planned.

[ittaiz]

Hey, I have this exact same need but I can't use short_path since we're talking about a macro...
It's basically wrapping rules_scala with a macro and computing some values (one of them being the short_path of the output of the rule being wrapped)

[ittaiz]

ping. I need this to be able to work with sandboxing/remote execution...

[hlopko]

Adding a link to a corresponding StackOverflow question.

[ulfjack]

It seems reasonable to change $(location) to output the runtime path for attributes that are clearly only used at runtime. If there are any cases where it's not clear, we could add another function $(rlocation) or $(runtime_location) to return the runtime path.

[ittaiz]

Another function would be great!

[ulfjack]

Do you think that there are a lot of cases where it's unclear?

[ittaiz]

Not at all. I have two use cases which both boil down to jvm_flags (one BUILD file and one in a macro). I'm just afraid a bit that changing the semantics would take longer than adding a new function.

[ulfjack]

I can think of use cases that require making this configurable by the user (genrule.cmd), but maybe we should also disallow $(location) for jvm_flags, where it's basically never correct (I think).

[ittaiz]

Why is this never correct? Or do you mean given a new runtime_location function then this is never correct?

…

On Wed, 24 May 2017 at 19:59 Ulf Adams ***@***.***> wrote: I can think of use cases that require making this configurable by the user (genrule.cmd), but maybe we should also disallow $(location) for jvm_flags, where it's basically never correct (I think). — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2475 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIFynqkF_gmCNXMkm5-oucWEK0ck85ks5r9GHUgaJpZM4L1oIQ> .

[shahms]

Banning $(location) in jvm_flags seems like the opposite of a solution.

[ulfjack]

jvm_flags are only applied at runtime, so a function that expands to the compile-time path is 'never' correct, except it happens to be ok for source files.

[ulfjack]

@shahms - the proposal is to add $(rlocation), replace all uses of $(location) in jvm_flags with $(rlocation), and then disallow using $(location) in jvm_flags (ideally with a good error message).

[shahms]

That makes more sense, thanks :-)

[lberki]

See b1b02d7 for prior art.

[buchgr]

Our documentation seems to suggest that file paths returned by $(location ...) are runtime paths

The expanded paths are relative to the runfiles directory of the *_test or *_binary rule.

https://bazel.build/versions/master/docs/be/make-variables.html#location

[lberki]

That's untrue at least for genrules. A quick glance at the code says that we get execpaths, which makes sense because otherwise the $(location) support in java_toolchain wouldn't make sense.

[ulfjack]

LocationExpander takes options, and can be configured to return runtime paths. :-/

…

On Wed, Jun 14, 2017 at 1:57 PM, lberki ***@***.***> wrote: That's untrue at least for genrules. A quick glance at the code says that we get execpaths, which makes sense because otherwise the $(location) support in java_toolchain wouldn't make sense. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2475 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHA9YUQbQxQNFLAQ0_gRmFBVbq7d6TUbks5sD8qcgaJpZM4L1oIQ> .

[lberki]

Yeah. Not from the BUILD file, though.

[ulfjack]

@buchgr just found out that this bug report is a bit vague in that it doesn't mention which rule you're using. java_binary actually expands the runtime location, not the compile-time location in jvm_flags, but scala_binary doesn't. The ctx.expand_location Skylark method calls into the code path that expands the compile-time location, and is not configurable. Unfortunately, if we need a Skylark API change, then this becomes more complex to implement. :-/

[ulfjack]

For example, say you have expansion/BUILD:

genrule(name = "foo", outs = ["foo.txt"], srcs = ["bar.txt"], cmd = "cp $< $@")

foo.txt:
execpath = bazel-out/darwin-fastbuild/bin/expansion/foo.txt
rootpath = expansion/foo.txt

bar.txt:
execpath = expansion/bar.txt
rootpath = expansion/bar.txt

The current $(location) function sometimes returns the exec path, and sometimes returns the root-relative path, depending on how the expander is configured (except you can't configure it through the skylark API, which has no knob for that).

[ittaiz]

I think I understand. It sounds like from skylark (due to the lack of knob) $(location) always works like $(rootpath)

…

On Wed, 25 Oct 2017 at 15:46 Ulf Adams ***@***.***> wrote: For example, say you have expansion/BUILD: genrule(name = "foo", outs = ["foo.txt"], srcs = ["bar.txt"], cmd = "cp $< $@") foo.txt: execpath = bazel-out/darwin-fastbuild/bin/expansion/foo.txt rootpath = expansion/foo.txt bar.txt: execpath = expansion/bar.txt rootpath = expansion/bar.txt The current $(location) function sometimes returns the exec path, and sometimes returns the root-relative path, depending on how the expander is configured (except you can't configure it through the skylark API, which has no knob for that). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2475 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIF7pfPXr9UhTCqaIwzdZW6Sd-IfPiks5svy2QgaJpZM4L1oIQ> .

[ulfjack]

Unfortunately there have been some delays due to travel and public holidays in Germany, as well as a commit gone wrong, which I had to revert. I'm currently rolling it forward again, and I have a few more changes on top of that, but they're not under review yet. It looks like we'll miss the 0.8.0 window, unfortunately.

[alexeagle]

When this does land - please also add testing for cross-workspace paths. If I use $(rootpath @other_wksp//:thing) I would expect external/other_wksp/thing or so

[davido]

Exception in thread "main" java.lang.NoSuchFieldError: ANNOTATION_PROCESSOR_MODULE_PATH

I'm seeing a similar failure when running Bazel java_tools tests on Windows: #4165. All is fine on Linux, though.

[rzhw]

Quick note that cross-workspace path tests were added in 702429b.