tools/test/test-setup.sh fails to collect TEST_UNDECLARED_OUTPUTS_DIR in a minimal remote execution environment

[dws]

Description of the problem / feature request:

Using bazel 3.1.0, I remotely executed a test that created a file in TEST_UNDECLARED_OUTPUTS_DIR. This failed as follows:

external/bazel_tools/tools/test/test-setup.sh: line 389: file: command not found
Could not create "/tmp/worker/shard/operations/49e83fda-5340-46c5-a27d-fa393f7c87ca/bazel-out/k8-fastbuild/testlogs/atg/experimental/user/dws/report/test.outputs/outputs.zip": zip not found or failed

It appears that (a) tools/test/test-setup.sh requires a file(1) command to be present, and that (b) our remote execution environment is minimal enough not to provide one.

The relevant fragment of code in test-setup.sh has portability provisions for stat, even handling the case of it not being there at all:

    386       # stat has different flags for different systems. -c is supported by GNU,
    387       # and -f by BSD (and thus OSX). Try both.
    388       file_size="$(stat -f%z "$undeclared_output" 2>/dev/null || stat -c%s "$undeclared_output" 2>/dev/null || echo "Could not stat $undeclared_output")"
    389       file_type="$(file -L -b --mime-type "$undeclared_output")"
    390

It would be great if it had similar provisions for file(1) and pretty much all the other external commands it relies on.

Feature requests: what underlying problem are you trying to solve with this feature?

I would like to remove obstacles to obtaining additional data from the remote execution of tests using bazel.

If the remote test execution mechanism could work without any system commands being present, that would be excellent.
Barring that, if the test encyclopedia could enumerate what the remote test execution environment must provide in order for bazel's internal scripting to work, that would give us something to work off of in configuring our remote execution environment.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Easy to say: set up a remote test executor that does not make any system commands available to the test runner. Then use it to remotely execute a test that leaves a file in TEST_UNDECLARED_OUTPUTS_DIR.

What operating system are you running Bazel on?

Ubuntu 18.04.

What's the output of bazel info release?

release 3.1.0

Have you found anything relevant by searching the web?

No.

@werkt

[allada]

This looks like the same issue as: #11556 ?

[ulfjack]

This one seems specifically about tools used in test-setup.sh. Adding a fallback if file doesn't work or is not installed seems reasonable to me. I think that this will require an external contributor willing to send a patch and audit the code.

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 2+ years. It will be closed in the next 14 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler". Please reach out to the triage team (@bazelbuild/triage) if you think this issue is still relevant or you are interested in getting the issue resolved.

[coeuvre]

Should be fixed by recent changes, see #17884.