py_binary target inside another Starlark rule's runfiles doesn't get built fully

[mzhaom]

I have a skylark "run_under" rule that essentially does:

bazel run --script_path "name" --run_under "under" "command"
Then run the script

The code is here:
https://github.com/mzhaom/trunk/blob/master/tools/build_rules/run_under.bzl

The behavior I would expect is:

• Build the "under" target
• Build the "command" target
• Produce the wrapper script
• Then run the wrapper script if it's a test

The issue I got is, if the "under" target is a py_binary, only the
wrapper script for the py_binary is produced, but not the runfiles
symlink tree, as a result, the py_binary can't be run.

I have a reproducible case here:

https://github.com/mzhaom/trunk/blob/master/examples/run_under/BUILD

If I do bazel clean then just run

$ bazel test examples/run_under:hello_test

I got error:

python2: can't open file
'/usr/local/home/_bazel_mzhao/c0fe0a5986a7dee82d17f872b1091a3e/trunk2/bazel-out/local_linux-fastbuild/bin/examples/run_under/env.runfiles/examples/run_under/env.py':
[Errno 2] No such file or directory

[hanwen]

AFAICT, your rule doesn't add the runfiles for 'under' (eg. 'env') to the runfiles of _run_under_impl.

[hanwen]

http://bazel.io/docs/skylark/rules.html -> "Rules"

[mzhaom]

What's the right way to access the 'under''s runfiles? I tried

runfiles=ctx.runfiles(
  files = runfiles,
  transitive_files = ctx.attr.under.transitive_runtime_files,
)

But got the error:

/usr/local/home/mzhao/g/trunk/tools/build_rules/run_under.bzl:42:26: target (rule class of 'py_binary') doesn't have provider 'transitive_runtime_files'.

[mzhaom]

I made a change my run_under and it works with an older version of bazel:
bazelment/trunk@8f2589b

But it didn't work with the latest bazel which has sandbox enabled:

examples/run_under/hello_test: line 3: /usr/local/home/_bazel_mzhao/7f4ef1d4b60d94ba335123843e5cccc0/trunk/bazel-out/local_linux-fastbuild/bin/examples/run_under/env: No such file or director

It looks like I might miss putting some file into certain dependency so the sandbox doesn't mount it properly. @philwo could you suggest a solution? Thanks!

[philwo]

I will have a detailed look tomorrow, but here some thoughts:

I don't think the path manipulation, where you process ctx.configuration.bin_dir into "build_directory" and then add the short_name of your "exe" and "under" artifacts gives you the correct path to the files in the runfiles.

You may be able to see the paths where the files are actually to be found if you pass --verbose_failures to "bazel run" when you run your example. The sandbox should then print a lot of debugging output, including a list of mounts. I think you can find your "env" binary in there and maybe with that, you can deduce a better way to get the path in your Skylark script.

I will try to run your example on my workstation tomorrow and see if I can find a solution that works. :)

[damienmg]

@philwo ping? Is that really a bug?

[ulfjack]

Runfiles, meh. There's a good chance that there's a bug here, or two.

[dslomov]

Actions API cleanup

[brandjon]

It's unclear if this is still an issue, and if so, whether it's caused by the use of the API or because there's a bug in --run_under.

[pcjanzen]

related: #11997

[ShreeM01]

Hi there! We're doing a clean up of old issues and will be closing this one. Please reopen if you’d like to discuss anything further. We’ll respond as soon as we have the bandwidth/resources to do so.