Test outputs should work with builds without the bytes

[allada]

Background

We have been implementing remote execution and focusing heavily on setting the system up so it can use BwoB for our CI infrastructure, however some of our more complex tests want to output meta information for Jenkins (like XML and other meta data revolving around TEST_UNDECLARED_OUTPUTS_DIR). Due to --remote_download_outputs=minimal it does not download the outputs.

Desired outcome

A way to use minimal downloads but download all test outputs (possibly a new command line flag?).

Workaround

--remote_download_outputs=all does download everything including the undeclared outputs directory data (toplevel does download what you'd expect but as a zip file). This is obviously less desirable since our full build is ~200G and minimal is ~9G.

Version

3.0.0

[ulfjack]

Have you tried --remote_download_toplevel?

[allada]

Yes as stated in Workaround section it toplevel does download the xml files but not the undeclared outputs.

toplevel also forces us to download 100+G of files that are not actually needed because 100% of our tests that we run in this configuration are able to run remotely and the binaries are not needed locally.

[ulfjack]

Duh! Sorry I missed that.

[ulfjack]

I tried this, and Bazel seems to download the XML files and the undeclared outputs, and does not download the binary:

$ bazel test --config=engflow --test_output=all --remote_download_toplevel --nobuild_runfile_links //javatest/com/engflow/example:ExampleTest

$ find bazel-testlogs/ -type f
bazel-testlogs/javatest/com/engflow/example/ExampleTest/test.outputs/outputs.zip
bazel-testlogs/javatest/com/engflow/example/ExampleTest/test.outputs_manifest/MANIFEST
bazel-testlogs/javatest/com/engflow/example/ExampleTest/test.log
bazel-testlogs/javatest/com/engflow/example/ExampleTest/test.xml
bazel-testlogs/javatest/com/engflow/example/ExampleTest/test.cache_status

However, if you run bazel build with the same flags, it does download the binary. The undeclared outputs end up zipped as test.outputs/outputs.zip. I think it would be nicer if they were pulled unzipped into a undeclared_outputs directory.

[allada]

Yep you are correct, I tested again and see it as a zip file. I have adjusted the wording in the opening post. In our case we can use toplevel and modify our tooling to handle the zip file. In the past toplevel did download more than we wanted, but I'll need to investigate further, since so much has been fixed/changed in our code base that we might not have the issues any more.