Feasibility? bounded by RAM more than by CPU: make sccache RAM pressure-aware

[marcusmueller]

tl;dr

no mechanism to delay spawning compile jobs when nearly out of RAM exists. The ninja/make level is sadly the wrong place to add that functionality. Can sccache be extended to do that?

Situation (a.k.a. the pain of the modern maintainer in AI hype times)

Due to a) rust and modern C++ compilation and linking being increasingly more memory-intense and b) server-grade RAM price increases over the last years, FOSS projects like GNU Radio now commonly find themselves in the bad situation that they don't get the resources they need – if I run ninja or make with -j$(ncpu), one of the compilation processes will end up getting OOM-killed (unless building happens on a machine with > 5GB RAM per CPU core, which is expensive to rent or buy) when being stuck building multiple large translation units simultaneously.

But: if I run with -j( memory / 5GB), then system CPU remains underutilized for the majority of time.

There is GNU make flags that stop make from launching compilers on high CPU load. Sadly, no such flags exist for low available memory.

Problem (i.e., what's sccache even to do with any of this?)

Telling the build tool to not even attempt to start a compilation job in a low-memory situation is a thinkable approach here – sadly, attempts to integrate that into ninja have repeatedly come to a halt.

The underlying problem, when combined with build artifact caching, i.e., with sccache, is that due to the server architecture of sccache, the ninja (or make) process starting the compilation is not aware of other, independent builds, as they commonly happen in CI environments. This problem becomes even harder when we're looking at distributed compilation!

Instead of delaying kicking off a build request due to low available RAM, it would hence be better if the delay decision happened at the build server level, i.e., within sccache. This is also advantageous in case of higher-latency caches being involved – you really don't want to delay checking/fetching available object files just because you "only" have 4 GB RAM available.

Proposed Approach

In sccache, only at the point when the decision that something can't be retrieved from cache has been made, check for available memory. If below a configurable threshold, wait for a future that encapsulates what is essentially a while RAM_available() < threshold { sleep(0.5); } launch_compiler() closure.

This could be done with a priority queue (so that we end up with a guaranteed first requested, first finished priority) or just with individual threads or async wait loops as described above.

Question on feasibility

Before I dive into that, is that even something that is considered architecturally feasible, or am I missing something important?