cg_llvm: Move target machine command-line quoting from C++ to Rust #146700

Zalathar · 2025-09-18T02:41:05Z

When this code was introduced in #130446 and #131805, it was complicated by the need to maintain compatibility with earlier versions of LLVM.

Now that LLVM 20 is the baseline (#145071), we can do all of the quoting in pure Rust code, and pass two flat strings to LLVM to be used as-is.

In this PR, my priority has been to preserve the existing behaviour as much as possible, without worrying too much about what the behaviour should be. (Though I did avoid a leading space before the first argument.)

rustbot · 2025-09-18T02:41:10Z

r? @lcnr

rustbot has assigned @lcnr.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Zalathar · 2025-09-18T02:43:49Z

cc @durin42 @nebulark @aeubanks (contributors to the C++ code being replaced)

Zalathar · 2025-09-18T02:50:40Z

@bors try jobs=x86_64-msvc-1,x86_64-msvc-2

cg_llvm: Move target machine command-line quoting from C++ to Rust try-job: x86_64-msvc-1 try-job: x86_64-msvc-2

rust-bors · 2025-09-18T05:16:21Z

💔 Test for bee730a failed: CI. Failed jobs:

try - x86_64-msvc-1 (web logs, extended logs)

Zalathar · 2025-09-18T05:20:31Z

Oh I see, setting Quote = true causes each argument to always be quoted, regardless of whether it contains any parts that would need quoting.

@rustbot author

rustbot · 2025-09-18T05:20:36Z

Reminder, once the PR becomes ready for a review, use @rustbot ready.

Zalathar · 2025-09-18T05:35:24Z

I've heard second-hand that this function shows up on perf results. I don't know whether that's true, but it can't hurt to check.

@bors try @rust-timer queue

cg_llvm: Move target machine command-line quoting from C++ to Rust

rust-bors · 2025-09-18T07:46:01Z

☀️ Try build successful (CI)
Build commit: 44b341a (44b341a1c7f27c26b78205e3264d584b69a9c53f, parent: 93117677d857bb7c3f12c9dc500d77839f8fb13d)

rust-timer · 2025-09-18T09:51:20Z

Finished benchmarking commit (44b341a): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.2%, 0.2%]	2
Regressions ❌ (secondary)	0.2%	[0.1%, 0.3%]	16
Improvements ✅ (primary)	-0.4%	[-0.5%, -0.2%]	4
Improvements ✅ (secondary)	-3.1%	[-23.4%, -0.2%]	15
All ❌✅ (primary)	-0.2%	[-0.5%, 0.2%]	6

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

Results (secondary -5.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.1%	[2.1%, 2.1%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-9.0%	[-9.4%, -8.6%]	2
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 471.663s -> 472.562s (0.19%)
Artifact size: 387.94 MiB -> 387.87 MiB (-0.02%)

Zalathar · 2025-09-19T00:24:06Z

There's more followup that could be done here, like doing this quoting only once (instead of per-CGU), or skipping it for non-PDB targets. But I think that just moving the quoting step from C++ to Rust is worth landing on its own.

@rustbot ready

nikic · 2025-09-19T08:41:41Z

@bors r+

bors · 2025-09-19T08:41:44Z

📌 Commit 8b0a254 has been approved by nikic

It is now in the queue for this repository.

bors · 2025-09-19T18:32:21Z

⌛ Testing commit 8b0a254 with merge 0be8e16...

bors · 2025-09-19T21:36:58Z

☀️ Test successful - checks-actions
Approved by: nikic
Pushing 0be8e16 to master...

github-actions · 2025-09-19T21:40:04Z

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 5904356 (parent) -> 0be8e16 (this PR)

Test differences

Show 2 test diffs

2 doctest diffs were found. These are ignored, as they are noisy.

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard 0be8e16088894483a7012c5026c3247c14a0c3c2 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

aarch64-apple: 5671.4s -> 6397.4s (12.8%)
dist-aarch64-apple: 6108.6s -> 6660.7s (9.0%)
x86_64-mingw-2: 7795.4s -> 8429.4s (8.1%)
dist-x86_64-windows-gnullvm: 4769.5s -> 5093.2s (6.8%)
dist-x86_64-apple: 7808.8s -> 7298.5s (-6.5%)
x86_64-gnu: 6956.3s -> 6581.8s (-5.4%)
x86_64-msvc-2: 7157.6s -> 6782.6s (-5.2%)
x86_64-msvc-ext3: 6312.6s -> 6627.0s (5.0%)
dist-ohos-armv7: 4152.6s -> 3958.7s (-4.7%)
dist-x86_64-mingw: 9413.0s -> 9046.7s (-3.9%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

rust-timer · 2025-09-19T22:46:41Z

Finished benchmarking commit (0be8e16): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.6%, -0.2%]	4
Improvements ✅ (secondary)	-3.0%	[-23.4%, -0.2%]	16
All ❌✅ (primary)	-0.4%	[-0.6%, -0.2%]	4

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

Results (secondary -4.8%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.5%	[2.5%, 2.5%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-8.4%	[-9.5%, -7.3%]	2
All ❌✅ (primary)	-	-	0

Binary size

Results (secondary 0.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.0%	[0.0%, 0.0%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Bootstrap: 471.972s -> 472.625s (0.14%)
Artifact size: 389.96 MiB -> 389.93 MiB (-0.01%)

…bzol Add self-profile events for target-machine creation These code paths are surprisingly hot in the `large-workspace` benchmark (e.g. see perf changes from rust-lang#146700), suggesting room for more improvement. It would be handy to see some detailed timings and execution counts.

Rollup merge of #146845 - Zalathar:prof-target-machine, r=Kobzol Add self-profile events for target-machine creation These code paths are surprisingly hot in the `large-workspace` benchmark (e.g. see perf changes from #146700), suggesting room for more improvement. It would be handy to see some detailed timings and execution counts.

cg_llvm: Move target machine command-line quoting from C++ to Rust When this code was introduced in rust-lang#130446 and rust-lang#131805, it was complicated by the need to maintain compatibility with earlier versions of LLVM. Now that LLVM 20 is the baseline (rust-lang#145071), we can do all of the quoting in pure Rust code, and pass two flat strings to LLVM to be used as-is. --- In this PR, my priority has been to preserve the existing behaviour as much as possible, without worrying too much about what the behaviour *should* be. (Though I did avoid a leading space before the first argument.)

…bzol Add self-profile events for target-machine creation These code paths are surprisingly hot in the `large-workspace` benchmark (e.g. see perf changes from rust-lang#146700), suggesting room for more improvement. It would be handy to see some detailed timings and execution counts.

Remove current code for embedding command-line args in PDB The compiler currently has code that will obtain a list of quoted command-line arguments, and pass it through to TargetMachine creation, so that the command-line args can be embedded in PDB output. This PR removes that code, due to subtle concerns that might not have been apparent when it was originally added. --- Those concerns include: - The entire command-line quoting process is repeated every time a target-machine-factory is created. In incremental builds this typically occurs 500+ times, instead of happening only once. The repeated quoting constitutes a large chunk of instructions executed in the `large-workspace` benchmark. - See #146804 (comment) for an example of the perf consequences of skipping all that work. - This overhead occurs even when building for targets or configurations that don't emit PDB output. - Command-line arguments are obtained in a way that completely bypasses the query system, which is a problem for the integrity of incremental compilation. - Fixing this alone is likely to inhibit incremental rebuilds for most or all CGUs, even in builds that don't emit PDB output. - Command-line arguments and the executable path are obtained in a way that completely bypasses the compiler's path-remapping system, which is a reproducibility hazard. - #128842 --- Relevant PRs: - #113492 - #130446 - #131805 - #146700 - #146973 Zulip thread: - https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/Some.20PDB.20info.20bypasses.20the.20query.20system.20and.20path.20remapping/with/541432211 --- According to #96475, one of the big motivations for embedding the command-line arguments was to enable tools like Live++. [It appears that Live++ doesn't actually support Rust yet](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/embeded.20compiler.20args.20and.20--remap-path-prefix/near/523800010), so it's possible that there aren't any existing workflows for this removal to break. In the future, there could be a case for reintroducing some or all of this functionality, guarded behind an opt-in flag so that it doesn't cause problems for other users. But as it stands, the current implementation puts a disproportionate burden on other users and on compiler maintainers.

Remove current code for embedding command-line args in PDB The compiler currently has code that will obtain a list of quoted command-line arguments, and pass it through to TargetMachine creation, so that the command-line args can be embedded in PDB output. This PR removes that code, due to subtle concerns that might not have been apparent when it was originally added. --- Those concerns include: - The entire command-line quoting process is repeated every time a target-machine-factory is created. In incremental builds this typically occurs 500+ times, instead of happening only once. The repeated quoting constitutes a large chunk of instructions executed in the `large-workspace` benchmark. - See rust-lang/rust#146804 (comment) for an example of the perf consequences of skipping all that work. - This overhead occurs even when building for targets or configurations that don't emit PDB output. - Command-line arguments are obtained in a way that completely bypasses the query system, which is a problem for the integrity of incremental compilation. - Fixing this alone is likely to inhibit incremental rebuilds for most or all CGUs, even in builds that don't emit PDB output. - Command-line arguments and the executable path are obtained in a way that completely bypasses the compiler's path-remapping system, which is a reproducibility hazard. - rust-lang/rust#128842 --- Relevant PRs: - rust-lang/rust#113492 - rust-lang/rust#130446 - rust-lang/rust#131805 - rust-lang/rust#146700 - rust-lang/rust#146973 Zulip thread: - https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/Some.20PDB.20info.20bypasses.20the.20query.20system.20and.20path.20remapping/with/541432211 --- According to rust-lang/rust#96475, one of the big motivations for embedding the command-line arguments was to enable tools like Live++. [It appears that Live++ doesn't actually support Rust yet](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/embeded.20compiler.20args.20and.20--remap-path-prefix/near/523800010), so it's possible that there aren't any existing workflows for this removal to break. In the future, there could be a case for reintroducing some or all of this functionality, guarded behind an opt-in flag so that it doesn't cause problems for other users. But as it stands, the current implementation puts a disproportionate burden on other users and on compiler maintainers.

rustbot assigned lcnr Sep 18, 2025

rust-bors bot added a commit that referenced this pull request Sep 18, 2025

Auto merge of #146700 - Zalathar:quoted-args, r=<try>

bee730a

cg_llvm: Move target machine command-line quoting from C++ to Rust try-job: x86_64-msvc-1 try-job: x86_64-msvc-2

This comment has been minimized.

Sign in to view

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 18, 2025

Move target machine command-line quoting from C++ to Rust

8b0a254

Zalathar force-pushed the quoted-args branch from adfa0b3 to 8b0a254 Compare September 18, 2025 05:25

This comment has been minimized.

Sign in to view

rust-bors bot added a commit that referenced this pull request Sep 18, 2025

Auto merge of #146700 - Zalathar:quoted-args, r=<try>

44b341a

cg_llvm: Move target machine command-line quoting from C++ to Rust

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Sep 18, 2025

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Sep 18, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 19, 2025

rustbot assigned nikic and unassigned lcnr Sep 19, 2025

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 19, 2025

bors added the merged-by-bors This PR was explicitly merged by bors. label Sep 19, 2025

bors merged commit 0be8e16 into rust-lang:master Sep 19, 2025
12 checks passed

rustbot added this to the 1.92.0 milestone Sep 19, 2025

This was referenced Sep 19, 2025

Weekly cargo update #145516

Open

Weekly cargo update + update allowed dependencies list #145849

Open

rustbot removed the perf-regression Performance regression. label Sep 19, 2025

Zalathar deleted the quoted-args branch September 19, 2025 23:17

Zalathar mentioned this pull request Sep 21, 2025

Add self-profile events for target-machine creation #146845

Merged

Zalathar mentioned this pull request Sep 24, 2025

Compute quoted args for debuginfo at most once per session #146973

Closed

Zalathar mentioned this pull request Sep 25, 2025

Remove current code for embedding command-line args in PDB #147022

Merged

Uh oh!

cg_llvm: Move target machine command-line quoting from C++ to Rust #146700

cg_llvm: Move target machine command-line quoting from C++ to Rust #146700

Conversation

Zalathar commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Sep 18, 2025

Uh oh!

Zalathar commented Sep 18, 2025

Uh oh!

Zalathar commented Sep 18, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors bot commented Sep 18, 2025

Uh oh!

Zalathar commented Sep 18, 2025

Uh oh!

rustbot commented Sep 18, 2025

Uh oh!

Zalathar commented Sep 18, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors bot commented Sep 18, 2025

Uh oh!

This comment has been minimized.

rust-timer commented Sep 18, 2025

Overall result: ❌✅ regressions and improvements - please read the text below

Uh oh!

Zalathar commented Sep 19, 2025

Uh oh!

nikic commented Sep 19, 2025

Uh oh!

bors commented Sep 19, 2025

Uh oh!

bors commented Sep 19, 2025

Uh oh!

bors commented Sep 19, 2025

Uh oh!

Uh oh!

github-actions bot commented Sep 19, 2025

Test differences

Job duration changes

Uh oh!

rust-timer commented Sep 19, 2025

Overall result: ✅ improvements - no action needed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Zalathar commented Sep 18, 2025 •

edited

Loading