bootstrap/miri: avoid rebuilds for test builds #143681

RalfJung · 2025-07-09T12:12:28Z

When building Miri in its own repo, we always build with --all-targets:

rust/src/tools/miri/miri-script/src/util.rs

Lines 167 to 174 in a009612

    
           // We build all targets, since building *just* the bin target doesnot include 
        
           // `dev-dependencies` and that changes feature resolution. This also gets us more 
        
           // parallelism in `./miri test` as we build Miri and its tests together. 
        
           let mut cmd = self 
        
               .cargo_cmd(crate_dir, "build", features) 
        
               .args(&["--all-targets"]) 
        
               .args(quiet_flag) 
        
               .args(args);

This saves a bunch of time since some of Miri's dependencies get more features enabled by some of Miri's dev-dependencies, and they all get built twice otherwise if you do cargo build && cargo test (which is typically what you end up doing inside ./miri test and also inside ./x test miri).

This applies the same approach to bootstrap, drastically reducing the edit-compile cycle for Miri work here. :)

rustbot · 2025-07-09T12:12:33Z

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

RalfJung · 2025-07-12T07:56:21Z

r? @jieyouxu @Kobzol I think you've been doing most of the reviewing in this area recently?

I guess the alternative would be to just not 'build' miri here

rust/src/bootstrap/src/core/build_steps/test.rs

Lines 545 to 548 in b0e4c35

    
           // Build our tools. 
        
           let miri = builder.ensure(tool::Miri { compiler: target_compiler, target: host }); 
        
           // the ui tests also assume cargo-miri has been built 
        
           builder.ensure(tool::CargoMiri { compiler: target_compiler, target: host });

since it'll anyway get built as part of the cargo test invocation. But that doesn't seem to be how bootstrap is structured so --all-targets is probably better.

Another alternative would be to magically always pass --all-targets when a build step is requested by a test step -- that would avoid this problem systematically for all tools, not just for Miri. I have no idea how to do that.^^

jieyouxu · 2025-07-12T08:23:13Z

I think this is fine, but I'm not sure if all tools want --all-targets, so it seems better scoped to miri only for now.

r? @Kobzol (second opinion)

rustbot · 2025-07-12T08:23:17Z

Kobzol is not on the review rotation at the moment.
They may take a while to respond.

Kobzol

since it'll anyway get built as part of the cargo test invocation. But that doesn't seem to be how bootstrap is structured so --all-targets is probably better.

I was just thinking about this recently, if it is possible to "just" cargo test rustc_private tools without prebuilding them first. If yes, then I think it would be nice to try it, it could reduce some edge cases and code in bootstrap. I haven't gotten around to taking a look at test yet though, still cleaning up build and check. But even if we did this, you'd still get rebuilds if you alternate x build miri and x test miri, right?

Btw, I think it's time to start thinking about dogfooding rust-lang/cargo#14774 in bootstrap, maybe that could help generally avoid these kinds of issues.

src/bootstrap/src/core/build_steps/tool.rs

Co-authored-by: Jakub Beránek <berykubik@gmail.com>

RalfJung · 2025-07-12T10:12:34Z

But even if we did this, you'd still get rebuilds if you alternate x build miri and x test miri, right?

Yes. But I rarely ever do that.

Kobzol · 2025-07-12T11:20:03Z

Ok. I think there might be a better general solution, but this is fine as an improvement for now. You can r=me once CI is green.

RalfJung · 2025-07-13T06:36:54Z

@bors r= Kobzol rollup

bors · 2025-07-13T06:36:57Z

📌 Commit b703451 has been approved by ``

It is now in the queue for this repository.

jieyouxu · 2025-07-13T06:40:11Z

@bors r=Kobzol (lol)

bors · 2025-07-13T06:40:13Z

📌 Commit b703451 has been approved by Kobzol

It is now in the queue for this repository.

RalfJung · 2025-07-13T06:44:22Z

Ah damn, missed the space... thanks!

jieyouxu · 2025-07-13T06:45:45Z

Ah damn, missed the space... thanks!

rust-lang/bors#353 for new bors to handle :P

…r=Kobzol bootstrap/miri: avoid rebuilds for test builds When building Miri in its own repo, we always build with `--all-targets`: https://github.com/rust-lang/rust/blob/a00961269107703772c4e8f071f0accbe0f1a7e5/src/tools/miri/miri-script/src/util.rs#L167-L174 This saves a bunch of time since some of Miri's dependencies get more features enabled by some of Miri's dev-dependencies, and they all get built twice otherwise if you do `cargo build && cargo test` (which is typically what you end up doing inside `./miri test` and also inside `./x test miri`). This applies the same approach to bootstrap, drastically reducing the edit-compile cycle for Miri work here. :)

Rollup of 16 pull requests Successful merges: - #142885 (core: Add `BorrowedCursor::with_unfilled_buf`) - #143217 (Port #[link_ordinal] to the new attribute parsing infrastructure) - #143355 (wrapping shift: remove first bitmask and table) - #143448 (remote-test-client: Exit code `128 + <signal-number>` instead of `3`) - #143592 (UWP: link ntdll functions using raw-dylib) - #143681 (bootstrap/miri: avoid rebuilds for test builds) - #143710 (Updates to random number generation APIs) - #143724 (Tidy cleanup) - #143820 (Fixed a core crate compilation failure when enabling the `optimize_for_size` feature on some targets) - #143850 (Compiletest: Simplify {Html,Json}DocCk directive handling) - #143855 (Port `#[omit_gdb_pretty_printer_section]` to the new attribute parsing) - #143868 (warn on align on fields to avoid breaking changes) - #143875 (update issue number for `const_trait_impl`) - #143881 (Use zero for initialized Once state) - #143887 (Run bootstrap tests sooner in the `x test` pipeline) - #143893 (Don't require `eh_personality` lang item on targets that have a personality) Failed merges: - #143878 (Port `#[pointee]` to the new attribute parsing infrastructure) - #143891 (Port `#[coverage]` to the new attribute system) r? `@ghost` `@rustbot` modify labels: rollup

…r=Kobzol bootstrap/miri: avoid rebuilds for test builds When building Miri in its own repo, we always build with `--all-targets`: https://github.com/rust-lang/rust/blob/a00961269107703772c4e8f071f0accbe0f1a7e5/src/tools/miri/miri-script/src/util.rs#L167-L174 This saves a bunch of time since some of Miri's dependencies get more features enabled by some of Miri's dev-dependencies, and they all get built twice otherwise if you do `cargo build && cargo test` (which is typically what you end up doing inside `./miri test` and also inside `./x test miri`). This applies the same approach to bootstrap, drastically reducing the edit-compile cycle for Miri work here. :)

Rollup of 17 pull requests Successful merges: - #142885 (core: Add `BorrowedCursor::with_unfilled_buf`) - #143217 (Port #[link_ordinal] to the new attribute parsing infrastructure) - #143355 (wrapping shift: remove first bitmask and table) - #143448 (remote-test-client: Exit code `128 + <signal-number>` instead of `3`) - #143681 (bootstrap/miri: avoid rebuilds for test builds) - #143710 (Updates to random number generation APIs) - #143724 (Tidy cleanup) - #143738 (Move several float tests to floats/mod.rs) - #143820 (Fixed a core crate compilation failure when enabling the `optimize_for_size` feature on some targets) - #143850 (Compiletest: Simplify {Html,Json}DocCk directive handling) - #143855 (Port `#[omit_gdb_pretty_printer_section]` to the new attribute parsing) - #143868 (warn on align on fields to avoid breaking changes) - #143875 (update issue number for `const_trait_impl`) - #143881 (Use zero for initialized Once state) - #143887 (Run bootstrap tests sooner in the `x test` pipeline) - #143893 (Don't require `eh_personality` lang item on targets that have a personality) - #143901 (Region constraint nits) Failed merges: - #143878 (Port `#[pointee]` to the new attribute parsing infrastructure) - #143891 (Port `#[coverage]` to the new attribute system) r? `@ghost` `@rustbot` modify labels: rollup

…r=Kobzol bootstrap/miri: avoid rebuilds for test builds When building Miri in its own repo, we always build with `--all-targets`: https://github.com/rust-lang/rust/blob/a00961269107703772c4e8f071f0accbe0f1a7e5/src/tools/miri/miri-script/src/util.rs#L167-L174 This saves a bunch of time since some of Miri's dependencies get more features enabled by some of Miri's dev-dependencies, and they all get built twice otherwise if you do `cargo build && cargo test` (which is typically what you end up doing inside `./miri test` and also inside `./x test miri`). This applies the same approach to bootstrap, drastically reducing the edit-compile cycle for Miri work here. :)

Rollup of 10 pull requests Successful merges: - #143217 (Port #[link_ordinal] to the new attribute parsing infrastructure) - #143681 (bootstrap/miri: avoid rebuilds for test builds) - #143724 (Tidy cleanup) - #143733 (Change bootstrap's `tool.TOOL_NAME.features` to work on any subcommand) - #143850 (Compiletest: Simplify {Html,Json}DocCk directive handling) - #143875 (update issue number for `const_trait_impl`) - #143881 (Use zero for initialized Once state) - #143887 (Run bootstrap tests sooner in the `x test` pipeline) - #143917 (Change "allocated object" to "allocation".) - #143918 (Tier check cleanup) r? `@ghost` `@rustbot` modify labels: rollup

rustbot assigned Mark-Simulacrum Jul 9, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Jul 9, 2025

This comment has been minimized.

Sign in to view

bootstrap/miri: avoid rebuilds for test builds

7fc2127

RalfJung force-pushed the bootstrap-miri-rebuilds branch from 8ad4dd3 to 7fc2127 Compare July 9, 2025 12:46

rustbot assigned jieyouxu and unassigned Mark-Simulacrum Jul 12, 2025

rustbot assigned Kobzol and unassigned jieyouxu Jul 12, 2025

Kobzol reviewed Jul 12, 2025

View reviewed changes

src/bootstrap/src/core/build_steps/tool.rs Outdated Show resolved Hide resolved

refine comment

b703451

Co-authored-by: Jakub Beránek <berykubik@gmail.com>

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 13, 2025

fmease mentioned this pull request Jul 14, 2025

Rollup of 16 pull requests #143912

Closed

jhpratt mentioned this pull request Jul 14, 2025

Rollup of 17 pull requests #143916

Closed

Kobzol mentioned this pull request Jul 14, 2025

Rollup of 10 pull requests #143919

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

bootstrap/miri: avoid rebuilds for test builds #143681

bootstrap/miri: avoid rebuilds for test builds #143681

Uh oh!

RalfJung commented Jul 9, 2025

Uh oh!

rustbot commented Jul 9, 2025

Uh oh!

This comment has been minimized.

RalfJung commented Jul 12, 2025

Uh oh!

jieyouxu commented Jul 12, 2025

Uh oh!

rustbot commented Jul 12, 2025

Uh oh!

Kobzol left a comment

Uh oh!

Uh oh!

RalfJung commented Jul 12, 2025

Uh oh!

Kobzol commented Jul 12, 2025

Uh oh!

RalfJung commented Jul 13, 2025

Uh oh!

bors commented Jul 13, 2025

Uh oh!

jieyouxu commented Jul 13, 2025

Uh oh!

bors commented Jul 13, 2025

Uh oh!

RalfJung commented Jul 13, 2025

Uh oh!

jieyouxu commented Jul 13, 2025

Uh oh!

Uh oh!

	// We build all targets, since building just the bin target doesnot include
	// `dev-dependencies` and that changes feature resolution. This also gets us more
	// parallelism in `./miri test` as we build Miri and its tests together.
	let mut cmd = self
	.cargo_cmd(crate_dir, "build", features)
	.args(&["--all-targets"])
	.args(quiet_flag)
	.args(args);

bootstrap/miri: avoid rebuilds for test builds #143681

Are you sure you want to change the base?

bootstrap/miri: avoid rebuilds for test builds #143681

Uh oh!

Conversation

RalfJung commented Jul 9, 2025

Uh oh!

rustbot commented Jul 9, 2025

Uh oh!

This comment has been minimized.

RalfJung commented Jul 12, 2025

Uh oh!

jieyouxu commented Jul 12, 2025

Uh oh!

rustbot commented Jul 12, 2025

Uh oh!

Kobzol left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

RalfJung commented Jul 12, 2025

Uh oh!

Kobzol commented Jul 12, 2025

Uh oh!

RalfJung commented Jul 13, 2025

Uh oh!

bors commented Jul 13, 2025

Uh oh!

jieyouxu commented Jul 13, 2025

Uh oh!

bors commented Jul 13, 2025

Uh oh!

RalfJung commented Jul 13, 2025

Uh oh!

jieyouxu commented Jul 13, 2025

Uh oh!

Uh oh!