Add a jobserver proxy to ensure at least one token is always held

[Zoxc]

This adds a jobserver proxy to ensure at least one token is always held by rustc. Currently with -Z threads rustc can temporarily give up all its tokens, causing cargo to spawn additional rustc instances beyond the job limit.

The current behavior causes an issue with cargo fix which has a global lock preventing concurrent rustc instances, but it also holds a jobserver token, causing a deadlock when rustc gives up its token. That is fixed by this PR.

Fixes #67385.
Fixes #133873.
Fixes #140093.

[rustbot]

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[jieyouxu]

r? @SparrowLii

[SparrowLii]

cc @matthiaskrgr

[SparrowLii]

Thanks!
@bors try @rust-timer queue

[bors]

⌛ Trying commit 141055f with merge c4cd906...

[bors]

☀️ Try build successful - checks-actions
Build commit: c4cd906 (c4cd906f6dc68c4343777d06a86285dbccb53e58)

[bjorn3]

cg_clif has https://github.com/rust-lang/rustc_codegen_cranelift/blob/master/src/concurrency_limiter.rs which I think does something similar to this PR.

[Zoxc]

@bjorn3 What's the purpose of ConcurrencyLimiter there? It's a bit unclear to me.

[rust-timer]

Finished benchmarking commit (c4cd906): comparison URL.

Overall result: ❌ regressions - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.4%	[0.4%, 0.4%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results (primary -0.7%, secondary -2.2%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.1%	[0.9%, 1.2%]	2
Regressions ❌ (secondary)	1.8%	[1.8%, 1.8%]	1
Improvements ✅ (primary)	-0.9%	[-1.7%, -0.5%]	12
Improvements ✅ (secondary)	-2.9%	[-4.6%, -1.9%]	5
All ❌✅ (primary)	-0.7%	[-1.7%, 1.2%]	14

Cycles

Results (primary -0.5%, secondary -2.8%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.5%	[-0.5%, -0.4%]	2
Improvements ✅ (secondary)	-2.8%	[-3.4%, -2.2%]	12
All ❌✅ (primary)	-0.5%	[-0.5%, -0.4%]	2

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 775.257s -> 775.095s (-0.02%)
Artifact size: 365.06 MiB -> 365.08 MiB (0.01%)

[bjorn3]

@bjorn3 What's the purpose of ConcurrencyLimiter there? It's a bit unclear to me.

It is responsible for requesting jobserver tokens and ensuring that the implicit token isn't returned to the jobserver. It also caches tokens that are expected to be used soon (it gets the total amount of cgus and only returns tokens if the amount of remaining cgus is less than the amount of tokens the compilation session has claimed in total).

[Zoxc]

I changed the code to use the jobserver's helper thread to help deal with a race which could leak a jobserver token when exiting the process.

[SparrowLii]

LGTM, cc @nnethercote @oli-obk since it changes some outer level functions

We can merge this if there are no other concerns

[nnethercote]

There isn't a single comment in the patch. At the very least, a comment on the GlobalCtxt::jobserver_proxy would be helpful for anyone who has to read this code in the future.

[SparrowLii]

@bors r+

[bors]

📌 Commit 08b27ff has been approved by SparrowLii

It is now in the queue for this repository.

[bjorn3]

Hang on, won't this somewhat misbehave if you run multiple compiler session inside the same process? They would each think they have an implicit token. There is a reason GLOBAL_CLIENT_CHECKED is a static, not a session field.

[Zoxc]

If you run them in parallel yes, but you should already be acquiring an extra token per rustc instance if you're using a jobserver in that case.

[bjorn3]

So you would need to use a separate proxy instance for acquiring the extra token per rustc instance?

[Zoxc]

No you'd use Client to get a token before creating the extra rustc instance. The proxy is for internal rustc use.

[bjorn3]

If you do it like the following you would deadlock with a jobserver that has 1 token available total:

for req in requests_stream {
    let token = acquire_token();
    spawn_rustc_instance(token);
}

You did have to do something like

let mut first_request = true;
for req in requests_stream {
    let token = if first_request {
        first_request = false;
        None
    } else {
        Some(acquire_token())
    };
    spawn_rustc_instance(token);
}

which is harder than making Proxy a static.

[Zoxc]

We could allow a Proxy to be passed in to rustc_interface to make that easier. Keep in mind Proxy cannot be a global as we need its destructor to run.