Possibility to disable caching for user code

[kmicklas]

Zig Version

0.10.0-dev.2783+76546b3f8

Steps to Reproduce

Build a very large binary with zig cc as the C toolchain. In our case, the binary is 584M.

Expected Behavior

Currently it is expected that the Zig cache may grow very large. However, when using an external build system to orchestrate the build and cache the results (in our case Bazel), we have no need for the Zig cache outside "internal" builds (libc stubs etc.). It would be great to have a flag to only cache these internal builds, and not user code.

Actual Behavior

With our 584M binary, the Zig cache is 761M after building from a clean cache.

[motiejus]

To add some light on the issue: when building some subtrees in Uber's go monorepo, the zig-cache promptly grows to hundreds of gigabytes, mostly duplicated with Bazel's cache. This may not be much for a single workstation, but that multiplies to a large number of engineers (which adds up to costs to CI and dev machines).

[nektro]

I think another route here could be for Zig to understand how to interact with Bazel's (or other) cache system(s) so that it can still know when to not do work and save on the cpu cycles, not just disk space

[kmicklas]

@nektro In our case I believe just not caching user code is sufficient. Bazel already caches user builds at the level of granularity that we care about. However we wouldn't want to turn off Zig's caching entirely because then we would waste cycles rebuilding smaller units that Bazel doesn't know about.

[motiejus]

There may be another way: instead of copying, try to reflink the files. If the setup is "careful enough", it will be faster and just save space. The "careful enough" is:

• supported file system. As of 5.19 that's btrfs, cifs, nfs, ocfs2, overlayfs and xfs.
• zig cache and the target directories are on the same mount point.

I will investigate this a bit more.

[motiejus]

There may be another way

Correction: it won't solve the Bazel caching issue fully, because bazel clean will not wipe that cache (as expected by Bazel's UX).

Copy-with-reflink, when available, is IMO a worthwhile optimization nevertheless.