Optimize sandbox performance

[jfirebaugh]

Link just the files needed to compile, link, or archive, rather than the entire directory tree. This drastically improves build times on macOS, where the sandbox is known to be slow (bazelbuild/bazel#8230).

Fixes #1026

[sbc100]

I this works I think it would be great improvement!

[sbc100]

What about the libraries (from the sysroot) that are used during linking? Are they handled separately?

[sbc100]

Is all_files even needed at all anymore?

[sbc100]

Where do tools/*.py and stuff like that get included?

[jfirebaugh]

I should say -- I don't have any deep knowledge of emscripten. My methodology here was just to start each of compiler_files, linker_files, ar_files with an empty list and add things until both local testing (on macOS) and my own project built successfully. So it is likely that dependencies are missing if they aren't covered by those cases.

What about the libraries (from the sysroot) that are used during linking? Are they handled separately?

I don't know. What libraries would those be and when would they be needed?

Where do tools/*.py and stuff like that get included?

What are those used for?

[sbc100]

I should say -- I don't have any deep knowledge of emscripten. My methodology here was just to start each of compiler_files, linker_files, ar_files with an empty list and add things until both local testing (on macOS) and my own project built successfully. So it is likely that dependencies are missing if they aren't covered by those cases.

What about the libraries (from the sysroot) that are used during linking? Are they handled separately?

I don't know. What libraries would those be and when would they be needed?

At link time you would access to all the system library in sysroot/libc/wasm32-emscripten. For example libc.a. The fact that your link is working without this makes me thing maybe the whole sysroot is still available somehow.

Where do tools/*.py and stuff like that get included?

What are those used for?

They are part of the emcc compiler. emcc.py depends a whole lot of other .py file such as the ones in tools/ and won't load without those. If emcc.py is running then somehow those files must be available.. but I can't see how you are making them available.

[jfirebaugh]

At link time you would access to all the system library in sysroot/libc/wasm32-emscripten. For example libc.a. The fact that your link is working without this makes me thing maybe the whole sysroot is still available somehow.

It looks like the answer is simply that nothing I've tested actually requires linking any of these libraries. For example, I can in fact remove the --sysroot argument entirely (by removing ACTION_NAMES.cpp_link* from the appropriate flag_set) and test_external still builds successfully.

They are part of the emcc compiler. emcc.py depends a whole lot of other .py file such as the ones in tools/ and won't load without those. If emcc.py is running then somehow those files must be available.. but I can't see how you are making them available.

Ah, this appears to be because python adds the containing directory of the script as the first entry in sys.path and if the script is a symlink, does so after resolving the symlink (I can't find this detail documented however). This is a sandbox bypass so I imagine bazel has found a workaround for it somewhere, but I'm not sure where to look.

[jfirebaugh]

It's an unresolved issue in Bazel: bazelbuild/bazel#7091

[walkingeyerobot]

Overall I'm good with this, but it looks like the windows CI for bazel failed here. It's possible something additional is needed for windows?

[jfirebaugh]

I don't know. As far as I can tell, bin/clang and bin/clang++ are correctly registered as inputs for compile actions, and I don't have access to a Windows environment to debug further. In case it was something related to unsandboxed builds, I tested bazel build //hello-world:hello-world-wasm --spawn_strategy=local locally, but that works.

[walkingeyerobot]

I don't have a windows set up for bazel at the moment. I'll try to get one soon to see if we can debug this break a bit easier.

[zaucy]

On windows all these executables need to have the .exe extension. A separate BUILD file specifically for windows may be necessary or maybe using build_file_content and template it.

[jfirebaugh]

Thank you for this hint! I've taken the templated build_file_content approach.

[sbc100]

I don't really understand how this change can function. It doesn't seem to include any of the files under lib/.. or tools/... many of which are needed for even simple emscripten linking.

For example, src/library.js .. this file is always needed during linking. As are all the other .js files in that directory.

[kjlubick]

@trybka, you had asked about a before and after benchmark. I noticed a substantial improvement when building CanvasKit on my 24 core Linux machine.

# emsdk-3.1.10
$ bazel clean && bazel build //modules/canvaskit:canvaskit_webgl --compilation_mode=dbg
...
INFO: Elapsed time: 6031.283s, Critical Path: 701.38s
INFO: 1662 processes: 101 internal, 1561 linux-sandbox.
INFO: Build completed successfully, 1662 total actions

# emsdk-3.1.10 with RAM disk
$ bazel clean && bazel build //modules/canvaskit:canvaskit_webgl --compilation_mode=dbg --sandbox_base=/dev/shm
...
INFO: Elapsed time: 179.974s, Critical Path: 75.22s
INFO: 1662 processes: 101 internal, 1561 linux-sandbox.
INFO: Build completed successfully, 1662 total actions

# emsdk-3.1.13
$ bazel clean && bazel build //modules/canvaskit:canvaskit_webgl --compilation_mode=dbg
...
INFO: Elapsed time: 215.952s, Critical Path: 71.56s
INFO: 1662 processes: 101 internal, 1561 linux-sandbox.
INFO: Build completed successfully, 1662 total actions

# emsdk-3.1.13 with RAM disk
$ bazel clean && bazel build //modules/canvaskit:canvaskit_webgl --compilation_mode=dbg --sandbox_base=/dev/shm
...
INFO: Elapsed time: 126.265s, Critical Path: 78.31s
INFO: 1662 processes: 101 internal, 1561 linux-sandbox.
INFO: Build completed successfully, 1662 total actions

With my sandbox on a RAM disk - a 30% improvement. Using the default sandboxing, a ~30x improvement.

However, I do notice some issues when trying to compile the release mode and with a remote executor. Not all the files are specified, e.g. the closure compiler. I can get a PR up that addresses those issues for me.