Consider building rust with Cargo

[alexcrichton]

In #2237 it was resolved to rewrite the build system in Rust, but that never ended up happening. It was later decided that we should build Rust with rustpkg, but that also ended up not happening. As a natural continuation of said issue we should consider building Rust with Cargo.

There are certainly some difficulties in doing this, some of which I can think of are:

• If cargo isn't built as part of the build process, changing the meaning of CLI flags may be tricky (over time)
• Is cargo's configuration flexible enough to encode the bootstrapping process?
• Is going through this change worth the effort?

This is most certainly a low priority issue, I believe building servo with Cargo to be a better milestone than building Rust.

[omasanori]

Indeed it might be P-low but I'm really excited to think of building rustc with cargo. 😄

[alexchandel]

Extracting rustc from rust-lang/rust into its own repository is a step towards this, and is probably easier to accomplish. It would make building Rust from source slightly more complicated, and it would simplify this repository's Metroid-like dependency graph:

[huonw]

The dependency graph would be equally complicated (likely unchanged), just spread over several repositories. I find it likely that splitting rustc into a separate repository while still doing lots of breaking & generally major changes as we are now would be a recipe for significant pain.

[alexchandel]

@huonw I was referring to this repository's dependency graph. Naturally the global situation would be unchanged. Disentangling rustc from rust is probably important in the long run, since rustc depends on several extra libraries outside the standard library (e.g. graphviz) that should not necessarily be versioned with the language. One difficulty that comes to mind is ensuring that the latest rustc is compatible with the latest rust, but this is manageable with continuous integration.

And having libstd as a dependency of rustc might make sense from a cargo standpoint, assuming cargo can incorporate the snapshot/bootstrapping process.

[huonw]

I'm not exactly sure why simplifying the dependency graph of a single repository is valuable, when the overall dependency graph of the officially supported crates is not changing; splitting it across many repositories just makes it harder to reason about, since it's now completely spread out. (My only point is we should not be looking at this from the point-of-view of repositories' dependency graphs; just focus on whatever other benefits it gives.)

One difficulty that comes to mind is ensuring that the latest rustc is compatible with the latest rust, but this is manageable with continuous integration.

It's not immediately obvious that it's easy; we'd certainly need to enhance bors significantly, and people submitting changes may need to submit and keep track of pull requests to upto n different repositories with them all reviewed and ready to merge at once (i.e. bors would have to collect all associated PRs together to land in one go... we'd even need a way to associate them reliably, both for bors and for historical purposes in the commit log).

[Gankra]

@alexcrichton can you give us an update on the efforts to get this done? Some stuff flared up in the summer, but I dunno what came of it.

[jonas-schievink]

@gankro That was #27003

I don't think all of the limitations in Cargo (whichever these were) were addressed since then. Specifically, the PR depended on rust-lang/cargo#1802.

[alexcrichton]

In addition to the links that @jonas-schievink I'm also working on this on an external branch. My current stance is that I would like to avoid two build systems at all costs, so the only way to land this would be to entirely obsolete the current build system, which is taking some time to do :).

[alexcrichton]

Done!

rustbuild is tracked here -- #31590